ASM Build Spec

Canonical source: https://github.com/Clocktower-and-Associates/ASM. The copy on clocktowerassoc.com mirrors that repository.

Implementation Standard for the Agentic Web

Author: Wesley Shoffner, Clocktower and Associates Version: 1.1 Date: August 2026

---

Changes from v1.0

Version 1.1 is a revision of the March 2026 release. The manifest schema, the markup vocabulary, the three-tier model, and the readiness levels are unchanged, so a manifest written against v1.0 remains valid. The changes are to the argument, the supporting evidence, and several statements of fact.

1. Corrected a factual claim about vendor user-agent strings. Version 1.0 stated in Section 2.1 and again in Section 6.5 (NT-4) that AI vendors used a single user-agent token for both their training crawlers and their user-facing agents, and it named OpenAI's Operator and Anthropic's ClaudeBot as examples. That was true when it was written and it is no longer true. OpenAI, Anthropic, and Perplexity now publish distinct tokens per activity. Section 2.1 has been rewritten to state this plainly, to record the tokens with links to the vendor documentation, and to rebuild the case for agent_policy on grounds that survive the change.

2. Rewrote Section 2.1 and the closing argument of Section 2.3. The "block everything or allow everything" framing and the assertion that robots.txt offers no middle ground have been removed. The case for declaring agent policy in agents.json now rests on three stated grounds: what robots.txt cannot express at all, the fact that token-to-activity mapping is a per-vendor convention rather than a standard, and the compliance asymmetry documented by the vendors themselves for user-directed traffic.

3. Added a forward-looking note on identification at the end of Section 2.1, covering the unauthenticated nature of user-agent strings and the direction of signed-request proposals such as Cloudflare's Web Bot Auth.

4. Replaced the conflict-resolution example in Section 2.4. The previous example turned on Crawl-delay, which is not part of RFC 9309 and which Google's documentation lists as unsupported. The section now says so and uses a conflict between max_requests_per_minute in agent_policy and a path-level rule in robots.txt instead.

5. Added a conformance caveat to Section 3.2. React-based server-rendering frameworks cannot emit a bare HTML comment into the document head. Layer 2 plus the supplementary Link response header is now stated to be conforming, and the reference implementation is described.

6. Rewrote NT-4 in Section 6.5 to match the corrected Section 2.1, including a table of current vendor tokens and the two caveats that apply when governing user-directed traffic through robots.txt.

7. Removed the rolling eighteen-month window in Section 1. The two questions are now stated without a moving deadline.

8. Expanded the standards comparison in Section 9. Added rows for WebMCP and for MCP server cards, and expanded the llms.txt row to record the file's canonical root location and to note that the reference site publishes both files.

9. Added a canonical-source line identifying the ASM repository as the authoritative copy of this document.

Version 1.0 remains published as history and is not withdrawn. Audit reports issued against v1.0 remain valid, since no scoring criterion or manifest requirement changed.

---

1. The Two Questions

Two questions now confront every website, and as of 2026 neither is hypothetical:

1. Will your site still work when there is no screen and no human?
2. Does your site allow autonomous agents to transact while still blocking the traffic you don't want?

ASM (Agent Site Manifest) answers both.

ASM is an implementation standard for building websites that AI agents can effectively discover, navigate, understand, and operate. Where the ASM Scoring Framework defines what to measure and how to score agent-readiness, ASM defines how to build for it.

ASM provides:

1. Traffic Governance: A three-tier model that separates crawlers from agents, giving site owners granular control over automated access.
2. An agent manifest: agents.json, a machine-readable declaration of a site's agent-facing capabilities and access policy.
3. A markup vocabulary: Data attributes that communicate intent, structure, and interaction contracts to agents beyond what HTML and ARIA alone provide.
4. Readiness levels: A tiered adoption path from quick wins to full agent optimization.

ASM is additive. It does not replace HTML semantics, ARIA, schema.org, or any existing web standard. It extends them where gaps exist.

---

2. Traffic Governance: The Three-Tier Access Model

2.1 What robots.txt Can and Cannot Express

robots.txt dates to 1994, when Martijn Koster proposed the method that RFC 9309 later documented as the Robots Exclusion Protocol. That RFC defines three rule types and no others: User-agent, Allow, and Disallow. The vocabulary was designed for a web in which the only automated visitor worth governing was an indexing crawler, and it still reflects that assumption.

Version 1.0 of this specification argued that robots.txt could not express the three-tier distinction at all, on the grounds that each vendor shipped a single user-agent token covering both its training crawler and its user-facing agent. That claim was accurate when it was written and it is not accurate now. In the intervening period the major vendors have published separate tokens for separate activities, and their own documentation now draws the line that ASM drew:

Vendor documentation: OpenAI, Anthropic, Perplexity.

We read that as convergence rather than as a refutation. The distinction ASM identified in v1.0, between traffic that harvests a site and traffic that serves a person who chose the site, is the distinction the vendors have since built into their own crawler fleets. A site owner who wants to refuse training collection while leaving user-directed fetches alone can now write that intent into robots.txt for the three vendors above, and should. Per-tier allow and deny is, for those vendors, partially expressible in robots.txt today.

The case for declaring agent policy in agents.json rests on three things that survive the change.

Expressiveness. robots.txt grants or refuses access to a path for a token, and that is the whole of its grammar. It has no standard way to state a rate ceiling, no way to say that reading a catalog is permitted while placing an order is not, no way to name the actions a site supports or the entry point for each one, and no way to point an agent at the authentication it will need before attempting a task. Those are the questions an arriving agent actually has, and agents.json answers all of them in a single fetch.

Portability of the declaration. The mapping from token to activity is a per-vendor convention published in vendor documentation and revisable by the vendor. Nothing standardizes it. There is no registry, no naming rule, and no obligation on a new entrant to follow the pattern that OpenAI, Anthropic, and Perplexity happen to share. A site owner who governs access through tokens alone therefore has to maintain a working catalog of every vendor's current fleet and revisit it whenever a vendor adds, renames, or retires an agent. Declaring agent_policy states the policy once in terms of what the traffic is doing, which is the thing the site owner has an opinion about, and leaves the mapping to the agent, which already knows which of its own tokens it is using.

Compliance asymmetry. This is the strongest of the three, and it applies precisely where ASM's interest is greatest. Anthropic's crawler documentation states that "Anthropic's Bots respect 'do not crawl' signals by honoring industry standard directives in robots.txt," and it lists Claude-User, the token used for user-initiated fetches, among those bots. OpenAI's documentation, describing ChatGPT-User, states that "because these actions are initiated by a user, robots.txt rules may not apply." Perplexity's documentation is more direct still: of Perplexity-User it states that "since a user requested the fetch, this fetcher generally ignores robots.txt rules." For the user-directed tier, then, robots.txt is binding on one of the three major vendors by their own account and explicitly non-binding on the other two. A site owner who writes a Tier 3 rule into robots.txt has written a rule that two of the three have said in advance they may disregard.

That gap is what agent_policy is built for. Where a directive is not binding, the instrument that actually works is not a stronger prohibition but a richer and more cooperative declaration, detailed enough that an agent honoring it completes its task faster and more reliably than an agent ignoring it. A Disallow line can only permit or refuse a fetch, whereas a manifest can set out the terms under which the fetch is welcome, which is a proposition an agent has a reason to accept.

A note on identification. Everything described above rests on a user-agent string, which is an unauthenticated self-declaration that any client can copy. Cloudflare's Web Bot Auth proposal addresses this by having agents sign their requests using HTTP Message Signatures (RFC 9421), and Cloudflare's write-up notes that "user agent headers alone are easily spoofed and are therefore insufficient for reliable identification" (Cloudflare). If identification does move toward signed requests, robots.txt has no way to consume the result, because its grammar has nowhere to put a key, a signature agent, or a condition on either. A policy layer does have somewhere to put them: agent_policy is a JSON document, and a future version of this specification can make signed identity a condition of the terms it grants without changing how the manifest is discovered or fetched.

2.2 The Three Tiers

ASM defines three tiers of automated web traffic, each with different access governance.

Tier 1: Crawlers (Indexing, Training, RAG Retrieval)

Automated systems that traverse and index content at scale: search engine crawlers, AI training pipelines, RAG retrieval systems.

Access governed by: robots.txt

This is what robots.txt was designed for. Tier 1 traffic MUST respect robots.txt directives. No change to existing conventions. ASM takes no position that overrides robots.txt for crawling traffic.

Tier 2: Discovery & Comparison Agents

Agents performing research or comparison tasks on behalf of a user, such as "find me the best price across ten sites," "compare features of these three products," "what restaurants near me have outdoor seating?"

These agents act on user intent but behave like crawlers: visiting many pages, extracting data, and moving on. They occupy a gray zone between indexing and user interaction.

Access governed by: agents.json agent policy

If agents.json is present, Tier 2 agents SHOULD follow its agent policy directives (rate limits, access rules) and MAY disregard robots.txt for the purpose of fulfilling user-initiated requests. If agents.json is absent, Tier 2 agents SHOULD fall back to respecting robots.txt.

If agent_policy.tier2_allowed is explicitly set to false, Tier 2 agents MUST NOT access the site.

Tier 3: User-Directed Task Agents

Agents executing a specific task on behalf of a user who has explicitly chosen to interact with the site, such as "buy this item," "fill out this contact form," "book an appointment for Tuesday."

The user has directed the agent to a specific site for a specific purpose. Blocking the agent blocks the user.

Access governed by: agents.json agent policy for rate limits and politeness. NOT bound by robots.txt.

Tier 3 agents are functionally equivalent to the user operating a browser. They SHOULD respect agent policy rate limits and politeness directives. They SHOULD NOT be blocked by robots.txt, because the intent is user-directed site interaction, not automated crawling.

If agent_policy.tier3_allowed is explicitly set to false, Tier 3 agents MUST respect this and inform the user that the site does not permit agent-mediated interaction.

2.3 The Strategic Value

The three-tier model gives a site owner one place to state a policy that would otherwise have to be assembled from several:

agents.json functions as the site's published terms for automated traffic. It tells any agent that reads it what the site is willing to have done, at what rate, under what conditions, and through which entry points, in a form that does not have to be rewritten when a vendor renames a bot or a new vendor appears.

Used alongside robots.txt, the division of labor is straightforward. robots.txt remains the right place to refuse bulk collection, and because vendors now publish per-activity tokens it can do that for those vendors without also refusing the user-directed traffic that brings customers. agent_policy carries the things robots.txt has no grammar for, and it carries the site's position on the one tier where, by two of the three major vendors' own documentation, a robots.txt rule is advisory rather than binding.

The practical effect is that access policy stops being a purely defensive setting maintained by whoever last touched the web server, and becomes a published, machine-readable statement that legal, security, and business teams can all read and agree on.

2.4 Conflict Resolution

The two files govern different things, so genuine conflicts are narrower than they might appear.

Rate is not among the things robots.txt can settle. RFC 9309 defines only User-agent, Allow, and Disallow, and contains no rate-limiting directive. Crawl-delay is a non-standard extension that some vendors choose to honor, Anthropic's crawler documentation among them, while Google's robots.txt documentation states that fields other than user-agent, allow, disallow, and sitemap are not supported. A site owner cannot rely on Crawl-delay reaching any particular agent, which is one of the reasons agent_policy carries rate limits instead.

The conflict that does arise in practice is between a path rule in robots.txt and a permission or ceiling in agent_policy. Consider a site whose robots.txt contains:

User-agent: Claude-User
Disallow: /checkout

and whose manifest contains:

"agent_policy": {
  "max_requests_per_minute": 30,
  "tier3_allowed": true
}

These are not in contradiction, and an ASM-aware agent SHOULD read them together. The tier3_allowed permission and the 30-request ceiling govern how the agent behaves across the site, while the Disallow narrows where it may go. A path-level refusal in robots.txt SHOULD be honored even where agent_policy permits the tier, and the max_requests_per_minute ceiling applies whether or not robots.txt says anything about rate.

Where the two genuinely disagree, for example a robots.txt that disallows a user-directed token entirely against an agent_policy that sets tier3_allowed: true, ASM-aware agents SHOULD follow the more restrictive of the two for Tier 1 and Tier 2 traffic. For Tier 3, an agent that can identify itself as user-directed, on a site that has opted into the tiered model by publishing agents.json, SHOULD follow the tier permission in agent_policy. Publishing the manifest is itself the site owner's statement that these tiers are being governed deliberately, and a blanket token-level rule written before a vendor split its tokens should not be read as a decision about a tier the site owner had no way to address separately at the time.

2.5 Future Considerations

The current model governs access at the site level. Path-level granularity, such as "Tier 2 allowed for product pages but not for account pages," is a recognized need that will be addressed in a future specification version. The site-level model is sufficient for initial adoption and covers the majority of current use cases.

---

3. The Agent Manifest: agents.json

3.1 Overview

agents.json is a machine-readable file hosted at /.well-known/agents.json, following the same convention as other .well-known resources, that declares a site's agent-facing capabilities, navigation structure, and access policy. A single file serves both ASM (site manifest) and ACP (service protocol) content.

It answers the question every agent has upon arriving at a site: "What can I do here, and how do I do it?"

This is the minimum viable adoption of ASM. A site can reach Level 1 readiness (see Section 7) by deploying agents.json and reviewing robots.txt. No HTML changes. No new attributes. One file.

3.2 Location & Discovery

The manifest MUST be accessible at:

https://example.com/.well-known/agents.json

Primary Discovery: Dual-Layer <head> Approach

Current LLM-based agents process the full HTML of a page, including comments. A dual-layer approach in the document <head> ensures discovery by both today's agents and tomorrow's:

<!-- AI agents: check /.well-known/agents.json for site navigation and operation instructions -->
<link rel="agents-manifest" type="application/json" href="/.well-known/agents.json">

Layer 1: The HTML Comment (Works Today)

A natural language directive like AI agents: check /.well-known/agents.json for site navigation and operation instructions is immediately understood by any language model parsing the page. It requires no prior training, no convention recognition, no ecosystem buy-in. This works with every LLM-based agent that can read HTML, right now.

Layer 2: The <link> Tag (Long-Term Standard)

<link rel="agents-manifest" type="application/json" href="/.well-known/agents.json">

This follows established web conventions (<link rel="manifest"> for PWAs, <link rel="sitemap"> for sitemaps, <link rel="icon"> for favicons). It's the correct long-term mechanism: machine-parseable, semantic, and consistent with how the web handles discoverability. Once agent providers begin checking for rel="agents-manifest" natively, this becomes the primary discovery path.

Transition Path:

Caveat: frameworks that cannot emit Layer 1. React-based server-rendering frameworks have no way to place a bare HTML comment in the document head. React's element model has no comment node, and the head components these frameworks expose, such as the Head component in next/document, reject dangerouslySetInnerHTML because they already receive children. Wrapping the directive in a <template> or <noscript> element would emit markup this specification does not describe, which is worse than omitting it. A site built on such a framework SHOULD therefore publish Layer 2 in the head and add the Link response header listed under Supplementary Discovery Mechanisms below. That combination is conforming, and it is what our own reference implementation does: the <link rel="agents-manifest"> tag is emitted from the document head, and Link: </.well-known/agents.json>; rel="agents-manifest" is set on responses in middleware.

Supplementary Discovery Mechanisms

These can supplement the <head> approach but are less reliable for current LLM-based agents:

• robots.txt directive: Agents: /.well-known/agents.json (mirrors the Sitemap: convention). Useful if agents check robots.txt before fetching pages.
• HTTP response header: Link: </.well-known/agents.json>; rel="agents-manifest". Doesn't require touching HTML, but agents must inspect headers.
• .well-known path: The file is already at /.well-known/agents.json, following IETF convention.

The <head> dual-layer approach is recommended as the primary mechanism because it places the directive inside content that agents are already reading.

3.3 Manifest Structure

The agents.json file serves both ASM (site manifest) and ACP (service protocol) content in a single file. A website with no API populates only site. A headless API populates only services. A full product populates both. One fetch, one parse, complete agent context.

The full manifest below shows every available field in the site block. Most are optional. A Level 1 manifest requires only version, site.name, site.description, site.capabilities, site.navigation, and site.agent_policy, roughly 25 lines of JSON. See Section 8.1 for the minimal quick-start template.

{
  // Required. agents.json format version.
  "version": "1.0",

  // ASM content: site-level manifest (see ASM spec).
  "site": {
    // Required. Human-readable site identity.
    "name": "Acme Corp",
    "description": "Enterprise widget manufacturing and distribution.",
    "primary_language": "en",
    "contact": "https://www.acme.com/contact",
    "sameAs": [
      "https://www.linkedin.com/company/acme-corp",
      "https://github.com/acme-corp"
    ],

    // Required. What agents can do on this site.
    "capabilities": {
      "actions": [
        {
          "id": "search_products",
          "description": "Search the product catalog by keyword, category, or specification.",
          "entry_point": "/products",
          "method": "form_submit",
          "input_schema": {
            "query": { "type": "string", "required": true, "description": "Search terms" },
            "category": { "type": "string", "required": false, "enum": ["widgets", "gizmos", "components"] }
          }
        },
        {
          "id": "get_quote",
          "description": "Request a price quote for a product configuration.",
          "entry_point": "/quote",
          "method": "form_submit",
          "input_schema": {
            "product_id": { "type": "string", "required": true },
            "quantity": { "type": "integer", "required": true, "min": 1 }
          },
          "authentication_required": true
        },
        {
          "id": "contact_sales",
          "description": "Submit a message to the sales team.",
          "entry_point": "/contact",
          "method": "form_submit",
          "input_schema": {
            "name": { "type": "string", "required": true },
            "email": { "type": "email", "required": true },
            "message": { "type": "string", "required": true }
          }
        }
      ],

      "data_types": [
        {
          "type": "product",
          "description": "Product listings with specifications, pricing, and availability.",
          "schema": "https://schema.org/Product",
          "locations": ["/products", "/products/*"]
        },
        {
          "type": "faq",
          "description": "Frequently asked questions about products and services.",
          "schema": "https://schema.org/FAQPage",
          "locations": ["/faq", "/support/faq"]
        }
      ]
    },

    // Required. Site structure for agent navigation.
    "navigation": {
      "sections": [
        {
          "name": "Products",
          "path": "/products",
          "description": "Full product catalog with specifications and pricing.",
          "children": [
            { "name": "Widgets", "path": "/products/widgets" },
            { "name": "Gizmos", "path": "/products/gizmos" },
            { "name": "Components", "path": "/products/components" }
          ]
        },
        {
          "name": "Pricing",
          "path": "/pricing",
          "description": "Pricing plans and volume discount schedules."
        },
        {
          "name": "About",
          "path": "/about",
          "description": "Company history, team, and mission."
        },
        {
          "name": "Contact",
          "path": "/contact",
          "description": "Sales inquiries, support requests, and office locations."
        }
      ],
      "sitemap": "/sitemap.xml"
    },

    // Optional. Authentication and access requirements.
    "access": {
      "public_content": true,
      "authentication": {
        "required_for": ["get_quote", "order_placement"],
        "methods": ["session_cookie", "api_key"],
        "login_url": "/login"
      }
    },

    // Optional. Technical context for agent interaction.
    "technical": {
      "rendering": "ssr",
      "spa_framework": null,
      "content_survivability": "full",
      "api_available": false,
      "api_documentation": null
    },

    // Optional. Agent access policy (see Section 2: Traffic Governance).
    "agent_policy": {
      "crawl_delay_seconds": 1,
      "max_requests_per_minute": 30,
      "tier2_allowed": true,
      "tier3_allowed": true
    }
  },

  // ACP content: service-level protocol (see ACP spec). Optional.
  "services": {
    // See ACP specification for structure.
  }
}

3.4 Manifest Fields Reference

All fields below are within the site block of agents.json. The services block is defined by the ACP specification.

site identity fields (Required)

site.capabilities.actions (Required, array)

Each action declares something an agent can do on the site.

site.capabilities.data_types (Optional, array)

Declares what structured data agents can extract.

site.navigation.sections (Required, array)

Hierarchical site structure for agent orientation.

site.access (Optional)

site.technical (Optional)

site.agent_policy (Optional)

Access rules and politeness directives for agent traffic. See Section 2 for the three-tier access model.

---

4. Design Principles

4.1 Progressive Enhancement

ASM layers on top of existing best practices. A site that uses semantic HTML, implements schema.org, and maintains a clean robots.txt is already partially agent-ready. ASM adds agent-specific capabilities without requiring sites to abandon or replace anything they already do.

4.2 Graceful Degradation

Every ASM addition degrades gracefully for user agents that don't understand it. Data attributes are ignored by browsers. The manifest is a separate file that only agents request. No ASM pattern breaks the human experience.

4.3 Discoverability Over Convention

Agents should not need to guess. ASM prefers explicit declaration over implicit convention. If a site has an agent-relevant capability, ASM provides a way to declare it unambiguously.

4.4 Minimal Surface Area

ASM targets the smallest vocabulary that addresses real gaps. It does not duplicate what semantic HTML, ARIA, and schema.org already solve. Every ASM pattern justifies its existence by addressing a capability gap identified by the ASM Scoring Framework.

---

5. Markup Vocabulary: data-asm-*

ASM introduces a set of data-asm-* attributes for cases where semantic HTML and ARIA don't provide sufficient signal for agent interaction. These attributes are supplementary: they never replace proper semantic markup.

These attributes are optional. A site can achieve Level 1 and Level 2 readiness (see Section 7) without using any data-asm-* attributes. The markup vocabulary is a Level 3 optimization for organizations that want maximum agent capability.

5.1 Page-Level Attributes

Applied to the <body> or top-level container. Tells the agent what kind of page this is and what it can accomplish here.

<body data-asm-page-type="product-listing"
      data-asm-page-purpose="Browse and compare widget products by category and specification.">

5.2 Section-Level Attributes

Applied to content sections to communicate intent and priority.

<section data-asm-role="primary-content"
         data-asm-summary="Product specifications and pricing for the X100 Widget.">
  ...
</section>

<aside data-asm-role="supplementary"
       data-asm-summary="Related products and cross-sell recommendations.">
  ...
</aside>

5.3 Action Attributes

Applied to interactive elements to communicate intent beyond what ARIA provides. The data-asm-consequences attribute is the most novel addition: nothing else in the existing standards ecosystem tells an agent what happens when it clicks a button.

<button data-asm-action="add-to-cart"
        data-asm-target="product:X100"
        data-asm-consequences="Item added to cart. Cart count increments. Cart drawer opens."
        aria-label="Add X100 Widget to cart">
  Add to Cart
</button>

<a href="/pricing"
   data-asm-action="navigate"
   data-asm-intent="view-pricing">
  See Pricing
</a>

5.4 Data Attributes

Applied to data-bearing elements to communicate structure and semantics.

<div data-asm-data-type="pricing-table"
     data-asm-data-key="product:X100"
     data-asm-data-freshness="2026-02-15">
  <dl>
    <dt>Base Price</dt>
    <dd data-asm-data-field="base_price"
        data-asm-data-value="299.99"
        data-asm-data-unit="USD">$299.99</dd>
    <dt>Volume Discount (100+)</dt>
    <dd data-asm-data-field="volume_price_100"
        data-asm-data-value="249.99"
        data-asm-data-unit="USD">$249.99</dd>
  </dl>
</div>

5.5 State Attributes

Applied to elements whose state affects agent decision-making but isn't covered by ARIA state attributes.

<div data-asm-state="in-stock"
     data-asm-state-label="In Stock (ships within 2 business days)">
  <span class="stock-badge green">●</span> In Stock
</div>

5.6 Visibility Attributes

Applied to elements that exist in the DOM but should be treated differently by agents.

<div class="cookie-banner" data-asm-visibility="dismiss"
     data-asm-dismiss-action="click:#accept-cookies">
  We use cookies...
  <button id="accept-cookies">Accept</button>
</div>

<!-- When the dismiss target IS this element, use click:self -->
<button class="promo-close" data-asm-visibility="dismiss"
        data-asm-dismiss-action="click:self">
  ✕ Close
</button>

<div class="promo-overlay" data-asm-visibility="ignore">
  Sign up for 10% off!
</div>

---

6. Remediation Patterns

Each pattern maps to an ASM dimension and addresses a specific failure mode. Patterns are ordered from highest to lowest impact within each dimension.

6.1 Content Survivability Patterns

CS-1: Server-Side Render Critical Content

If the site uses a client-rendered SPA, ensure critical business content (product information, pricing, contact details, primary CTAs) is server-rendered or statically generated. Use SSR frameworks (Next.js, Nuxt, SvelteKit, Astro) or pre-rendering.

CS-2: Meaningful <noscript> Fallbacks

For client-rendered sections, provide <noscript> content that communicates the purpose of the section and directs agents to alternative data sources.

<noscript>
  <p>This section requires JavaScript to display interactive product configurator.
  Product specifications are available at <a href="/products/x100/specs">/products/x100/specs</a>.</p>
</noscript>

CS-3: Declare Rendering Strategy

Include rendering in agents.json technical section. Agents can adjust their strategy based on whether the site is SSR, CSR, SSG, or hybrid.

6.2 Structural Legibility Patterns

SL-1: Landmark Coverage

Every page MUST have: <header>, <nav>, <main>, <footer>. Use <aside> for sidebars, <article> for self-contained content, <section> for thematic grouping. Label landmarks with aria-label when multiples of the same type exist.

<nav aria-label="Primary navigation">...</nav>
<nav aria-label="Footer navigation">...</nav>

SL-2: Sequential Heading Hierarchy

Every page MUST have exactly one <h1>. Subsequent headings follow sequential order without skipping levels.

✓ h1 → h2 → h3 → h3 → h2 → h3
✗ h1 → h3 → h5 (skipped levels)
✗ h1 → h1 (duplicate h1)

SL-3: Semantic Ratio Target

Target a semantic-to-generic element ratio above 15%. Replace layout <div> elements with <header>, <nav>, <main>, <section>, <article>, <aside>, <footer> wherever the element has a structural purpose.

SL-4: Language Declaration

Always set lang attribute on <html>. For multilingual content, set lang on content-level elements that differ from the page language.

6.3 Interactive Manifest Clarity Patterns

IMC-1: Native Interactive Elements

Use <button> for actions, <a> for navigation, <input>/<select>/<textarea> for data entry. If custom elements are necessary, add role, tabindex="0", and keyboard event handlers.

IMC-2: Accessible Name Coverage

Every interactive element MUST have an accessible name. Priority order: visible text content > <label> association > aria-label > aria-labelledby. Accessible names should describe the action, not the appearance ("Submit contact form" not "Blue button").

IMC-3: State Attributes on Stateful Controls

All controls with state MUST communicate that state via ARIA:

IMC-4: ASM Action Declarations (Level 3)

For primary actions, add data-asm-action and data-asm-consequences to give agents explicit understanding of what an interaction will do.

IMC-5: aria-live for Dynamic Content

Regions that update dynamically MUST have aria-live="polite" or aria-live="assertive" so agents can detect changes without polling.

6.4 Data Extractability Patterns

DE-1: Semantic Data Elements

Use <table> for tabular data (with <thead>, <th>, <caption>), <dl> for key-value pairs, <ol>/<ul> for lists. Never use <div> grids as a substitute for data tables.

DE-2: Schema.org Structured Data

Implement JSON-LD structured data for all business-critical content. Minimum coverage:

DE-3: Machine-Readable Values

Dates MUST use <time datetime="...">. Prices MUST be in text (not images). Use data-asm-data-value and data-asm-data-unit where the display format differs from the machine-parseable value. (Level 3)

DE-4: Consistent Data Patterns

Repeated content types (product cards, listing items, search results) MUST use consistent DOM structure. An agent that learns the pattern for one product card should be able to extract data from all product cards.

6.5 Navigation Traversability Patterns

NT-1: XML Sitemap

Publish a comprehensive XML sitemap at /sitemap.xml. Reference it in robots.txt. Update it when content changes.

NT-2: Breadcrumb Navigation

Implement breadcrumbs on all pages below the homepage. Mark up with BreadcrumbList schema.

NT-3: Static Link Coverage

All primary navigation paths MUST use standard <a href="..."> links that resolve without JavaScript. Hamburger/mobile menus must have <a> tags on every item even when the menu is visually collapsed.

NT-4: AI Crawler Access & Agent Policy

Review robots.txt for Tier 1 (crawler) access, and publish agents.json with an agent policy for the traffic robots.txt cannot govern.

The major vendors now publish a distinct user-agent token for each activity, so a robots.txt can separate bulk collection from user-directed fetching for those vendors:

Google publishes Google-Extended as a separate control for Google's AI products. Verify each token against the vendor's own documentation before relying on it, since the list changes.

Two caveats apply when governing the user-directed tier through robots.txt. First, the mapping from token to activity is a vendor convention rather than a standard, so the table above is a snapshot that has to be maintained. Second, and more consequential, the vendors do not agree on whether robots.txt binds their user-directed fetchers at all: Anthropic's documentation states that its bots honor robots.txt and lists Claude-User among them, OpenAI's states that robots.txt rules may not apply to ChatGPT-User because the action is user-initiated, and Perplexity's states that Perplexity-User generally ignores robots.txt rules. See Section 2.1 for the full argument and the source links.

Publish agent_policy so that the site's position on Tier 2 and Tier 3 traffic exists in a form those agents are asked to read, and so that rate limits, conditional permissions, capabilities, and entry points, none of which robots.txt can express, are available to any agent that arrives.

NT-5: Descriptive Link Text

Links MUST use descriptive text that communicates the destination or action.

NT-6: Pagination Over Infinite Scroll

Content lists MUST be navigable via standard paginated links. Infinite scroll or "load more" may be offered as an enhancement but must not be the only access pattern.

6.6 Agent Response Fitness Patterns

ARF-1: Information-First Content Order

In DOM order, the most important business information should appear early. Site purpose should be identifiable within the first 200 words of the text stream.

ARF-2: Descriptive Headings

Headings MUST summarize the content that follows. An agent reading only the headings should be able to produce an accurate page outline.

ARF-3: Image Alt Text

Informational images MUST have alt text that describes the information the image conveys, not the image's appearance. Decorative images MUST have alt="" and aria-hidden="true".

ARF-4: ASM Visibility Directives (Level 3)

Mark non-content overlays (cookie banners, promo modals, chat widgets) with data-asm-visibility="dismiss" or data-asm-visibility="ignore".

ARF-5: Eliminate Duplicate Content

Avoid duplicating heading or CTA text in aria-label attributes when the visible text is already sufficient.

---

7. Readiness Levels

ASM defines three readiness levels, providing a progressive adoption path. Each level has a clear effort boundary and a clear value proposition.

Level 1: Manifest & Access (Quick Wins: Days, Not Weeks)

What you do: Deploy /.well-known/agents.json. Review and fix robots.txt. No HTML changes required.

Why it matters: Agents arriving at your site immediately have a structured map of your capabilities, navigation, and access policy. You gain control over which automated traffic is welcome and which isn't. Sites that were previously blocking all AI traffic can now welcome agent customers while continuing to block training crawlers.

Expected ASM score improvement: +5–15 points on Navigation Traversability. Primary value is agent orientation and traffic governance rather than score improvement.

Level 2: Structural Readiness (HTML Hygiene: What You Should Already Be Doing)

What you do: Fix the HTML fundamentals. Semantic landmarks, heading hierarchy, native interactive elements, schema.org structured data. This is standard web development best practice: if your site doesn't already do these things, it has bigger problems than agent readiness.

Why it matters: Agents can now identify your page structure, find your interactive elements, and extract structured data. The site goes from "parseable with heuristics" to "machine-readable by design."

Expected ASM score at Level 2: 70–89 (Grade B)

Level 3: Agent-Optimized (Full ASM: Competitive Advantage)

What you do: Add data-asm-* attributes to communicate action intent, data structure, state, and visibility directives to agents. This is the optimization layer for organizations that want maximum agent capability and are investing in the agentic web as a strategic channel.

Why it matters: Agents don't just understand your site, they can operate it with confidence. They know what happens when they click a button, what data they can extract, what state an element is in, and what content to skip. This is the difference between "an agent can probably figure out your site" and "an agent can reliably transact on your site."

Expected ASM score at Level 3: 90–100 (Grade A)

---

8. Implementation Guide

8.1 Quick Start (5 Minutes to Level 1)

Step 1: Create /.well-known/agents.json. Start with this minimal template:

{
  "version": "1.0",
  "site": {
    "name": "Your Company",
    "description": "What your business does in one sentence.",
    "primary_language": "en",
    "capabilities": {
      "actions": [
        {
          "id": "contact",
          "description": "Send a message to our team.",
          "entry_point": "/contact",
          "method": "form_submit"
        }
      ]
    },
    "navigation": {
      "sections": [
        { "name": "Home", "path": "/" },
        { "name": "Products", "path": "/products" },
        { "name": "About", "path": "/about" },
        { "name": "Contact", "path": "/contact" }
      ],
      "sitemap": "/sitemap.xml"
    },
    "agent_policy": {
      "tier2_allowed": true,
      "tier3_allowed": true,
      "max_requests_per_minute": 30
    }
  }
}

Step 2: Review your robots.txt. If you are blocking GPTBot, ClaudeBot, or another AI user-agent, decide whether that is intentional, and check the token against the vendor's current documentation, since most vendors now publish a separate token for user-directed fetches (see Section 6.5, NT-4). Set your position on the crawling tier there, and use agent_policy for the rate limits, permissions, and entry points robots.txt cannot express.

Step 3: Verify /sitemap.xml exists and is current.

You are now Level 1 ASM-ready.

8.2 For Developers

Audit your templates. Most sites are generated from a small number of templates. Fix the template, fix every page that uses it.

Schema.org is the highest-ROI investment. It improves ASM Data Extractability scores and makes your content more useful to any system that consumes structured data: search engines, AI models, and agents alike.

Don't remove JavaScript, augment the baseline. ASM doesn't require abandoning client-side rendering. It requires ensuring critical content is available without it. SSR/SSG/hybrid approaches let you keep your architecture while meeting Content Survivability requirements.

8.3 For Framework Authors

Ship semantic defaults. Component libraries should render semantic HTML by default (<button> not <div role="button">).

Generate agents.json from route definitions. Frameworks with file-based routing (Next.js, Nuxt, SvelteKit, Astro) already have the navigation structure, so exposing it as an ASM manifest is straightforward. An @asm/next plugin that reads the app/ directory, generates agents.json, adds data-asm-page-type from route naming conventions, and infers data-asm-action from form elements would get sites to 80% of Level 3 readiness with a single npm install.

Provide ASM helpers. Consider an <AsmAction> wrapper component that adds data-asm-action and data-asm-consequences attributes.

8.4 For CMS Platforms

Include agents.json generation in site build pipelines. Content types map to capabilities.data_types. Navigation structures map to navigation.sections.

Default robots.txt should allow AI crawlers. Many CMS platforms ship with restrictive defaults that block GPTBot, ClaudeBot, etc.

Schema.org should be automatic. Generate JSON-LD from content type definitions and field mappings.

---

9. Relationship to Existing Standards

---

10. Relationship to ASM Scoring Framework

The Scoring Framework identifies the gap. The Build Spec closes it. Together they form the diagnostic-and-treatment framework for the agentic web.

---

Versioning

ASM is a living specification. Changes are versioned and backward-compatible within major versions.

• v1.0 (March 2026): Initial release. Three-tier traffic governance, the agents.json manifest, the data-asm-* markup vocabulary, and three readiness levels.
• v1.1 (August 2026): Revision. Corrected the account of vendor user-agent tokens, rebuilt the argument for agent_policy, replaced the conflict-resolution example, added a conformance caveat for React-based frameworks, and expanded the standards comparison. No change to the manifest schema, the markup vocabulary, or the readiness levels. See "Changes from v1.0" above.

The version field in agents.json declares the manifest format version, which remains "1.0" under this revision because the schema did not change.

---

License and Attribution

The ASM Build Spec is developed and maintained by Wesley Shoffner at Clocktower and Associates.

The specification is published for industry adoption. Framework authors, CMS platforms, and development teams are encouraged to implement ASM patterns and contribute feedback.

For audit services, implementation consulting, or tooling: Clocktower and Associates