How to Make Your Website Agent-Ready in 2026

Published 2026-06-04 · Last reviewed 2026-06-04

“Agent-ready” is the SEO term of 2026, and the term hides a question harder than the marketing copy admits. Which layer of your site are you making ready, for which kind of agent, and against which piece of evidence that the work actually pays off?

This article walks through the layers that the public agent-readiness rubrics check, in plain language for a site owner who does not run their own crawler. Each layer is named with what it actually does, the honest read on whether the dominant runtime consumers have committed to using it, and a “ship it in under an hour” version for the impatient. At the end there is a 60-minute checklist and a short note on the layer the public rubrics do not yet score.

The honest framing up front: agent-readiness is plural. There is no single switch that makes a website “ready” for every agent class. The Cloudflare Agent Readiness score, the AgentReady.org open spec, and a handful of practitioner guides bucket the same handful of layers slightly differently, and each layer has a different evidence weight. The fastest way to lose credibility in 2026 is to bill any single file (an llms.txt, an AGENTS.md, a @handle on any network) as the thing that makes your site agent-ready. The actual answer is half a dozen small, well-known files, deployed honestly, plus an understanding of which ones the dominant consumers will fetch and which are still speculative.

What “agent-ready” actually means in 2026

The phrase “agent-ready” started as a marketing line in Cloudflare’s Agents Week (13 to 17 April 2026) and now appears in roughly the same shape across three places. Cloudflare’s own first-party score at isitagentready.com checks five buckets: Discoverability, Content, Bot Access Control, Capabilities, and Commerce. The community open specification at AgentReady.org gives each layer a stable code (AR-DISC-01, AR-CONT-02, and so on) and tries to keep the rubric provider-neutral. A handful of practitioner blogs describe the same layers in plain English without the scoring overlay.

All three converge on a similar list of files and headers a site can publish to declare itself “ready.” That list is the spine of this article. The differences between rubrics are small and worth knowing about. Cloudflare counts a verified bot-management posture (Bot Access Control) in its score because it owns one. AgentReady.org weights human-readable identity higher than Cloudflare does because Cloudflare’s rubric is partly about Cloudflare-owned features. Neither rubric is wrong, both are partial, and a site that scores well on one usually scores well on the other.

The piece that gets lost in the scoring is which agent class the work is actually for. Two classes matter in 2026 and they fetch different things. The first is the answer-engine agent: ChatGPT browsing, Perplexity, Google’s AI Overviews retrieval, Claude’s tool-using web fetch. These consume crawled content, sometimes a markdown variant, and increasingly a structured tool surface like WebMCP. They are what the SEO-flavored “agent-ready” advice is aimed at. The second is the interactive task agent acting on behalf of a person, who wants to find an expert, qualify them, and exchange messages. The interactive class fetches different things (Agent Cards, registries, social profiles) and is the class the public readiness scores do not yet rate.

Holding both classes in mind keeps the rest of the article honest. Most of the layers below help the answer-engine class, with a small slice useful to interactive agents. The layer the rubrics still do not check is the one the interactive class relies on most, and it is the subject of its own section near the end.

Layer 1: Link headers (RFC 8288), the already-shipped foundation

The base layer is the most boring one, and the only one that has been standardised for nearly a decade. RFC 8288, “Web Linking,” published by Mark Nottingham in October 2017, defines the HTTP Link: response header as a stable place to advertise machine-readable relationships between a URL and other resources. The header has been in production at scale since well before the agent era. The change in 2026 is that the agent-readiness rubrics now treat its presence as a baseline signal that a site is configured for machine discovery at all.

In practice, Link headers carry the boring but useful pointers: a canonical URL for a page that has trackable variants, an alternate reference to a feed or RSS endpoint, a manifest reference to a Web App Manifest, and increasingly a service-desc pointer to an OpenAPI or other machine-readable API description. None of this requires new code if the site already publishes the equivalent <head> elements. A static site generator that emits clean <link> tags can mirror the same relations into the HTTP response with a few lines of edge configuration.

The honest evidence on Link headers is straightforward and they are not speculative. Every well-behaved crawler reads them, every common CDN can emit them, and every agent-readiness rubric checks for at least a canonical and a manifest pointer. The pitfall is the obvious one: a header that points to a stale or non-existent resource is worse than no header at all. The audit pattern that catches this is to crawl the header targets quarterly and surface 404s as a build-time check. Treat Link headers the way you treat sitemap.xml: present, accurate, refreshed when the underlying resource moves.

If your site is on a modern CDN such as Cloudflare, Fastly, Vercel, or Netlify, Link headers are usually a one-line setting. They are the cheapest row on any readiness rubric and the most embarrassing one to miss.

Layer 2: llms.txt and the markdown variants

llms.txt is the file that gets the marketing attention and carries the weakest public evidence of working as advertised. The proposal landed on 3 September 2024 from Jeremy Howard and the team at Answer.AI: a small Markdown index at the root of your domain, listing the primary documentation pages and canonical FAQ in a format a language model can consume without parsing rendered HTML. Adoption sits around 10% of measured domains, with the curious detail that adoption is slightly higher among mid-traffic sites than the largest publishers. The honest evidence is covered in detail in Does llms.txt actually work?; the short version for this article is that the dominant runtime consumers either say they do not use it (Google) or have not publicly committed to using it (OpenAI, Anthropic, Perplexity), and the largest public analysis (SE Ranking, roughly 300,000 domains) found no measurable AI-citation lift from publishing one.

The practical recommendation in 2026 is unchanged from the dedicated article. If llms.txt costs you an hour and you can keep it accurate, ship one. Do not bill it to a client as the AI visibility line item, and do not stack a vendor-managed “AI optimization” wrapper on top of the plain Markdown convention. Stale entries that point to deleted pages are the credibility tax that bites when a model does fetch the file. Refresh quarterly, or whenever the underlying canonical pages change.

The two markdown variants worth knowing about are llms-full.txt and AGENTS.md, and they are very different things. llms-full.txt is the same convention scaled up: it inlines the body of each linked document so a model can fetch the entire content set in a single request. The same caveats apply, and it is useful primarily as a self-contained documentation export rather than a verified visibility lever. AGENTS.md is unrelated. It is a README convention for coding agents (Cursor, Claude Code, Codex, and similar tools) that documents build, test, and lint commands at the root of a code repository. It is stewarded by the Agentic AI Foundation under the Linux Foundation and used by more than 60,000 repositories. It is not a website convention, and it does not pair with llms.txt despite the frequent blog-post grouping. A site owner thinking about agent-readiness for a marketing site does not need AGENTS.md; a maintainer of an open-source repository absolutely does.

Layers 3 and 4: Agent Skills Discovery and WebMCP

Above the markdown layer sit two structured layers that move agents from reading a site to actually calling it. Both are early, both are worth understanding, and both are best handled by shipping the cheap version now and the real version when the standards stabilise.

Agent Skills Discovery is Cloudflare’s RFC, first published on 17 January 2026 and last updated on 12 March 2026 at version v0.2.0. The convention is simple. A site publishes a JSON file at /.well-known/agent-skills/index.json that lists the skills the site offers to agents, in a form modeled on the MCP tool schema but reachable over plain HTTP. An agent that has a question about whether your site can look up a fractional CFO by speciality, run an internal search across a documentation corpus, or fetch a structured availability calendar can pull the index, read the skill manifests, and decide whether to call one. The RFC is still a Draft. There is no mandatory adoption, no scoring engine that fails a site for missing the file, and no provider that has publicly committed to using it at runtime. Shipping it today is a low-cost bet on a convention that may stabilise; do not promise a customer it changes their citation rate.

WebMCP is the structured-tools layer at the level of the browser. The specification lives in the W3C Web Machine Learning Community Group as a Draft Community Group Report, which is explicitly not a W3C Standard and is not under the Agentic AI Foundation. A page registers tools against a browser JavaScript API (the exact property name is still settling as the spec moves through its Chrome origin trial), and an agent running in the same browser session, or a future runtime that proxies the API, can list those tools and call them with structured arguments. WebMCP first appeared behind chrome://flags/#enable-webmcp-testing in Chrome 146-series builds, and the official origin trial opened with Chrome 149 in early June 2026, the build to test against if WebMCP is interesting to you.

The honest read on both layers is the same. Agent Skills Discovery and WebMCP are the right shape for the agentic web of 2027 and 2028. They are pre-stable today, and the public scoring tools do not yet ding a site for missing them. A site owner with a documentation surface that already exposes a useful API can prototype a WebMCP integration during the Chrome 149 origin trial and ship a thin Agent Skills Discovery index alongside it. Nothing more is required this quarter. If your site is a marketing brochure with no genuine machine-callable surface, both layers are skippable until the standards mature; faking a skill manifest with no real backend behind it is the kind of cosmetic agent-readiness work that hurts trust more than it helps the score.

Layer 5: Content Signals Policy, and the rubrics that score the layers

The last layer in the rubric is the one that controls who is allowed to consume the rest. Cloudflare’s Content Signals Policy, published on 24 September 2025, extends robots.txt with a Content-Signal: directive that lets a site owner declare permitted uses for the content a crawler fetches. The three tokens that matter are search (the content may be used to build a traditional search index), ai-input (the content may be used as input to a generative model at retrieval time), and ai-train (the content may be used to train a model). A site that wants to be cited by an AI assistant but not absorbed into a training corpus would publish Content-Signal: search, ai-input and omit ai-train. A site that wants to opt out of generative use entirely declares noai in the same family.

Content Signals is a policy, not a score. The directive token is honored at the discretion of each crawler and is most reliable inside Cloudflare’s own enforcement layer. Outside of that, it is a declared preference. The reason the agent-readiness rubrics include it is not because compliance is guaranteed but because the act of declaring a content-usage stance is itself a signal of operational maturity, and a missing or contradictory policy is the row in the rubric that exposes drift between an organisation’s public AI position and its actual crawler controls. Pair the directive with a short /ai-policy page that says the same thing in English, so a human auditor can match the machine declaration to the corporate statement.

Two scoring surfaces aggregate all five layers in 2026 and a site owner should pick one. The first is Cloudflare’s own first-party Agent Readiness score at isitagentready.com, now surfaced inside the Cloudflare URL Scanner. Its five buckets, in their canonical order, are Discoverability, Content, Bot Access Control, Capabilities, and Commerce. Cloudflare publishes adoption rates rather than a single readiness average (most measured sites have a robots.txt, only a few percent declare AI-usage preferences); for a single headline number, the third-party scanner AgentGrade found the top-100 sites it measured averaged roughly 55% in May 2026, which sets expectations honestly: even the largest, best-resourced publishers are not running clean. The second is the AgentReady.org community open spec, which weights the same layers with slightly different coefficients and gives each row a stable code (AR-DISC-01 through AR-COMM-08) that is easier to reference in a backlog ticket. Either rubric is reasonable. A small or mid-sized publisher is best served by checking the Cloudflare score quarterly, treating it as a temperature read, and reserving any engineering time for the rows where the rubric and the dominant providers agree that work matters.

What the rubrics still do not check, plus the 60-minute version

The five layers above tell an agent that your site exists, what it offers, what it permits, and how to call it. They do not tell the agent which human or business is behind the site, or how to address that party in a way another agent can also use. That is the row the readiness rubrics still leave blank.

The shape of the missing row is becoming visible across several projects in 2026. The Agent Network Protocol (ANP) defines a did:wba identifier and a /.well-known/agent-descriptions JSON-LD file in which each agent description carries a human-readable name field. ENSIP-27 defines an Ethereum-anchored agent card schema at /.well-known/agent.json. The A2A protocol, latest v1.0.1 published on 28 May 2026, standardises an Agent Card at /.well-known/agent-card.json; the detail of that file is covered in How A2A Agent Cards work. Each of these takes a slightly different angle on the same gap. None of them, today, is checked by isitagentready.com or AgentReady.org. The honest read is that the rubrics will incorporate this row as the standards stabilise; the cheap position today is to publish one of the public Agent Card variants and revisit when the scores update.

The 60-minute version of the rest of the rubric is short and serves as the take-home checklist:

• Confirm canonical, manifest, and a service-desc Link header are present and the targets resolve. Crawl the targets, fail the build on 404s. (15 minutes the first time, near-zero ongoing.)
• Ship a hand-written llms.txt at the root, listing the half-dozen pages a model would actually want to read. Refresh quarterly. (20 minutes.)
• Publish an /.well-known/agent-skills/index.json with the one or two skills your site genuinely offers an agent, even if both are read-only lookups. Skip if there is no real backend. (15 minutes when the backend exists.)
• Add a Content-Signal: line to your robots.txt with the policy you want, and a matching /ai-policy page in English. (10 minutes.)
• Decide whether to participate in the Chrome 149 WebMCP origin trial (opened in early June 2026). If your site is documentation- or tool-heavy, the answer is probably yes; if it is marketing-only, the answer is probably “wait.”

That covers four of the five Cloudflare buckets. The fifth, Commerce, applies only if your site sells or invoices anything and is a separate piece of work (Stripe Machine Payments Protocol, AP2, or x402 depending on your customer mix). The identity row is the one left for the next section.

How this connects to Tobira

The five layers above all answer some version of the question “how does an agent read or call your site.” None of them answers the human-facing question that the interactive class of agent has to solve: who is the person or business behind this site, and how do their agent and mine exchange identity before either side commits anything real?

That is the layer Tobira is building. A Tobira @handle adds a human-readable address for the site agent representing a person or a company, a public profile other agents can qualify against, and a mutual-reveal step before identity is exchanged. It is complementary to llms.txt, WebMCP, and the rest of the rubric, not a replacement. A clean readiness scorecard tells an agent your site is reachable and what it offers; a @handle tells the agent who is on the other end and how to address them next time. For the longer version of the addressable-versus-readable distinction, see Why your AI agent needs a name, not a wallet address.

Takeaways

• “Agent-ready” in 2026 is plural, not a single file. The Cloudflare Agent Readiness score buckets it as Discoverability, Content, Bot Access Control, Capabilities, and Commerce.
• Layer 1 (RFC 8288 Link headers) is the boring, already-shipped base layer every rubric checks for. Make sure canonical, manifest, and a service-desc pointer are present and the targets resolve.
• Layer 2 (llms.txt) is cheap to ship and weak on evidence of citation lift. Ship one if it costs an hour, keep it accurate, do not bill it as the AI visibility lever.
• Layer 3 (Agent Skills Discovery, Cloudflare RFC v0.2.0) and Layer 4 (WebMCP) are early but worth a thin prototype, especially around the Chrome 149 WebMCP origin trial that opened in early June 2026.
• Layer 5 (Cloudflare Content Signals Policy, 24 Sep 2025) is a declared content-usage policy in robots.txt. Publish one that matches your actual position; document it on /ai-policy.
• The readiness rubrics do not yet check the human-readable identity row that interactive agents (the ones acting for a person) actually rely on. That row is filled by Agent Cards and @handle-style identity layers, not by llms.txt or WebMCP.
• Independent scans (AgentGrade) put the top-100 average around 55% in May 2026; Cloudflare itself reports adoption rates, not one readiness average. The honest target for a small or mid-sized publisher is “clean and accurate on the boring layers,” not a high score on every speculative one.

FAQ

Is “agent-ready” a single standard?

No. Agent-readiness is plural in 2026. The Cloudflare first-party score at isitagentready.com checks five buckets (Discoverability, Content, Bot Access Control, Capabilities, Commerce). The AgentReady.org community open spec weights the same layers slightly differently. There is no single file or vendor that is sufficient to make a site agent-ready across every rubric and every agent class.

Does shipping llms.txt make my site agent-ready?

No. llms.txt is one row in the Content bucket and the evidence on its citation impact is currently weak. The largest analysis (SE Ranking, roughly 300,000 domains) found no correlation between having llms.txt and AI citation frequency, and Google has publicly said it does not use the file. Ship one if it takes an hour and you can keep it accurate, but it is not the whole rubric.

Is WebMCP shipped in Chrome?

Not yet. WebMCP is a W3C Web Machine Learning Community Group Draft Community Group Report (explicitly not a W3C Standard), exposed through a browser JavaScript API. It first appeared behind chrome://flags/#enable-webmcp-testing in Chrome 146-series builds, and the official origin trial opened with Chrome 149 in early June 2026.

Is AGENTS.md the same as llms.txt?

No. AGENTS.md is a README convention for coding agents (Cursor, Claude Code, Codex, and similar tools) that documents build, test, and lint commands inside a code repository. It is stewarded by the Agentic AI Foundation under the Linux Foundation and used by more than 60,000 repositories. It is not a website convention, and it does not pair with llms.txt despite the frequent blog-post grouping.

Where does identity fit in the agent-readiness picture?

The current public rubrics (Cloudflare Agent Readiness, AgentReady.org) do not yet score the human-readable identity layer that interactive agents acting for a person rely on. That layer is filled by Agent Cards (A2A v1.0.x at /.well-known/agent-card.json; ENSIP-27 at /.well-known/agent.json), Agent Network Protocol DID-based descriptions, and @handle-style identity layers like Tobira. Expect the rubrics to add this row as the standards stabilise.

What is the cheapest version of agent-ready I can ship in an hour?

Confirm canonical and manifest Link headers and that the targets resolve (about 15 minutes). Publish a hand-written llms.txt (about 20 minutes). Add a Content-Signal: directive to robots.txt and a short /ai-policy page (about 10 minutes). Optionally prototype an /.well-known/agent-skills/index.json with one or two genuine skills (about 15 minutes). Total: about one hour.

Sources

• Cloudflare Blog, Cloudflare Agent Readiness score (Agents Week 2026): https://blog.cloudflare.com/agent-readiness
• isitagentready.com (Cloudflare URL Scanner, first-party tool): https://isitagentready.com
• AgentReady.org open specification: https://agentready.org
• AgentGrade, third-party agent-readiness scan (source of the top-100 ~55% figure, May 2026): https://agentgrade.com/agent-readiness
• W3C Web Machine Learning Community Group, WebMCP Draft Community Group Report: https://webmachinelearning.github.io/webmcp/
• Chrome for Developers, “WebMCP: an interface for building rich agentic web experiences” (Early Preview Program): https://developer.chrome.com/blog/webmcp-epp
• Cloudflare Blog, Agent Skills Discovery RFC v0.2.0 (17 Jan 2026, updated 12 Mar 2026)
• Cloudflare Blog, Content Signals Policy (24 Sep 2025): https://blog.cloudflare.com/content-signals-policy
• IETF RFC 8288, Web Linking (Mark Nottingham, October 2017): https://www.rfc-editor.org/rfc/rfc8288.html
• Jeremy Howard / Answer.AI, llms.txt proposal: https://llmstxt.org
• Agentic AI Foundation, Linux Foundation directed fund: https://www.linuxfoundation.org/press/announcing-agentic-ai-foundation
• AGENTS.md, README convention for coding agents: https://agents.md
• A2A Protocol specification, latest v1.0.1 (28 May 2026): https://a2a-protocol.org/latest/specification