ProcStack's AI Metadata Specification v0.2

Updated : August 1st 2025

This document defines custom metadata and link conventions used across the ProcStack project pages to expose structured data to bots, crawlers, and language models.

This specification includes JSON schema definitions and comprehensive discovery methods for AI systems.

Meta Tags

• ai:data-source - Points to a JSON file containing full structured data about this page or project.
• ai:data-manifest - Points to a master manifest file listing all available structured content on the site.
• ai:content-api - Points to a base URL where related AI-parsable JSON files or endpoints exist.
• robots: ai:json - Optional value in robots meta tag indicating that AI-parsable structured content exists for this page.

Link Tags

• <link rel="alternate" type="application/json" ...> - JSON representation of the full site or specific page.
• <link rel="data-manifest" ...> - Points to the master structured content manifest (site-wide JSON).
• <link rel="ai-meta-spec" ...> - Points to this specification document for reference.

Discovery Files

• data-manifest.json - Master manifest with all available data sources, schemas, and metadata
• llms.txt - LLM-friendly text summary following community standards
• sitemap.xml - Standard sitemap with lastmod dates for change detection
• robots.txt - Includes AI-specific directives and manifest references

JSON Data Schema

Individual Page Data (e.g., /bots/PageName.htm.json)

{
  "title": "string",
  "description": "string",
  "lastModified": "YYYY-MM-DD",
  "media": [
    {
      "type": "video|image",
      "src": "relative/path/to/media",
      "alt": "alt text",
      "caption": "media description"
    }
  ]
}
  

Site Content Manifest (/bots/siteContent.json)

{
  "[pageKey]": {
    "jsonURL": "absolute URL to page JSON",
    "lastModified": "YYYY-MM-DD",
    "title": "page title",
    "description": "page description",
    "media": [...],
    "content": "full HTML content",
    "pageURL": "absolute page URL",
    "relativeURL": "relative page path"
  }
}
  

Use Cases

• Allow AI crawlers to skip parsing raw HTML and use the structured JSON directly
• Enable automated indexing or analysis of content for search, LLM embeddings, or documentation purposes
• Help language models understand project structure, contributors, and page purpose
• Prevent excessive requests to the same web server

Example 1 - Site Root

URL : https://procstack.github.io/index.htm

<!-- AI/LLM Data Discovery -->
<meta name="ai:data-source" content="https://procstack.github.io/bots/siteContent.json">
<meta name="ai:data-manifest" content="https://procstack.github.io/data-manifest.json">
<link rel="alternate" type="application/json" href="https://procstack.github.io/bots/siteContent.json" title="Full JSON of all Page's Content Data">
<link rel="data-manifest" type="application/json" href="https://procstack.github.io/data-manifest.json" title="Data Sources Manifest" />
  

Example 2 - Individual Page

URL : https://procstack.github.io/ProjectsLinks/currentsOfWar.htm

<!-- AI/LLM Data Discovery -->
<meta name="ai:data-source" content="https://procstack.github.io/bots/ProjectsLinks_currentsOfWar.htm.json">
<meta name="ai:data-manifest" content="https://procstack.github.io/data-manifest.json">
<link rel="alternate" type="application/json" href="https://procstack.github.io/bots/ProjectsLinks_currentsOfWar.htm.json" title="Single Page Content Data">
<link rel="data-manifest" type="application/json" href="https://procstack.github.io/data-manifest.json" title="Data Sources Manifest" />
  

Example 3 - API End-Point

<meta name="ai:content-api" content="https://procstack.github.io/bots/">
  

Example 4 - Enhanced robots.txt

User-agent: *
Allow: /

# AI/LLM specific directives  
User-agent: GPTBot
Allow: /
User-agent: ChatGPT-User
Allow: /

# Data endpoints
AI-Data-Manifest: https://procstack.github.io/data-manifest.json
AI-Content-API: https://procstack.github.io/bots/
  

Discovery Workflow for AI Systems

1. Check robots.txt for AI-Data-Manifest directive
2. Fetch data-manifest.json for complete data source overview
3. Use siteContent.json for bulk content or individual page JSON for specific content
4. Parse llms.txt for human-readable content summary
5. Respect lastModified dates to avoid unnecessary re-crawling