Web Bot Auth: how your site tells real AI agents from spoofs

Published July 7, 2026 · Last reviewed July 7, 2026

More of your traffic is now agents, and a growing share of it lies about who it is. A scraper can set its user-agent header to say “ChatGPT” or “Google” in one line of code, and for years the only reply a site had was to guess from IP ranges that shift constantly. That guessing game is what Web Bot Auth is built to end.

Web Bot Auth is a way for an automated client to cryptographically prove which agent it is, on every request, instead of asking the site to trust a string anyone can forge. It is being pushed by Cloudflare and picked up by Google, and it is one of the cleaner answers to the question every site owner now has: of all the agents knocking, which ones are real, and which are just wearing a name.

This piece explains what Web Bot Auth does, who is shipping it, and, just as important, the two questions it deliberately does not answer: which human an agent ultimately acts for, and whether you wanted it to reach you at all.

What Web Bot Auth actually is

Web Bot Auth is a proposed IETF standard that lets an automated client sign the HTTP requests it sends, so the receiving site can check the signature and know which agent is really on the line.¹ It is built on an existing, published internet standard: RFC 9421, HTTP Message Signatures.² The mechanics are less exotic than they sound.

The agent holds a private key and publishes the matching public key at a known location, a directory served under its /.well-known/ path. Before it sends a request, it signs a few chosen parts of that request, such as the target and a validity window, using the private key. It attaches two headers: a Signature-Input header describing what was signed and which key was used, and a Signature-Agent header naming where the public key lives. The site receives the request, fetches the public key from that published directory, and verifies the signature. If the math checks out, the request genuinely came from the holder of that key. If it does not, the request is a forgery, whatever its user-agent string claims.¹

The keys in the documented examples use Ed25519, a modern elliptic-curve signature scheme whose keys and signatures are small enough to attach to every request without a meaningful performance cost.¹ The important shift is not the cryptography, which is standard. It is the model. A user-agent string is a claim. A signature is proof, checkable by anyone with the public key, that cannot be copied from one request to another or replayed after its validity window closes.

Why user-agent strings and IP allowlists stopped working

For most of the web’s history, a site had two crude ways to tell a real crawler from an impostor. The first was the user-agent string, the line of text a client sends to name itself. The problem is that this line is self-reported and trivial to fake; anyone can send a request that says it is Googlebot. The second was an IP allowlist, checking that a request claiming to be Google actually came from Google’s published address ranges, often confirmed with a reverse DNS lookup. That works, but it is brittle. Address ranges change, they have to be maintained per crawler, and they break the moment an agent runs from somewhere new, such as a user’s own device or a shifting pool of cloud IPs.

The agent wave broke this model faster than it was already breaking. The number of distinct automated clients hitting a typical site has multiplied, and many of them are AI browsing agents acting for a specific person in the moment, not steady crawlers running from a fixed data center. There is no stable IP block to allowlist for “an agent a user launched from a laptop in Lisbon.” At the same time, the incentive to impersonate a trusted name went up, because sites increasingly grant favorable treatment, or any access at all, to agents they recognize.

So the site owner is stuck between two bad options: block aggressively and turn away legitimate agents that could be customers, or allow generously and let spoofed traffic walk in wearing a borrowed name. Web Bot Auth exists to give a third option. Instead of inferring identity from where a request came from, verify it from what the request carries: a signature only the real agent could have produced.

Who is actually shipping it: Cloudflare, Google, and the IETF

This is not a whiteboard proposal. Cloudflare has been the loudest backer, framing Web Bot Auth as a way to move past IP-based verification. It switched on message-signature checking at its edge in 2025, folded that into its Verified Bots Program the same year, and kept expanding it through its 2026 Agents Week updates, so signed agents can be recognized across the sites Cloudflare sits in front of.³ Because a large slice of the web runs behind that edge, one provider adopting it puts the mechanism in reach of a lot of sites at once.

Google is the other name that matters here, because it moved from talking to shipping. Google added a distinct crawler identity, Google-Agent, for AI systems that browse the web on a user’s behalf, the kind of agent behind its Project Mariner browsing work. A subset of Google-Agent’s requests are signed with Web Bot Auth and authenticated as agent.bot.goog, with the signature verifiable against public keys published in that domain’s /.well-known/ directory.⁴ Two caveats are worth stating plainly, because Google states them: Googlebot, the classic search crawler, is not signing its requests this way yet, and Google does not sign every request even from the agent that does. Google’s own guidance is to keep using IP-range and reverse-DNS checks as a fallback while signed traffic rolls out gradually.⁴

Under both is an IETF effort to standardize the scheme, with participation from Cloudflare, Amazon, Akamai, OpenAI, and Google among others.¹ That matters because the value of a signature convention scales with how many parties agree on it. If every large agent operator publishes keys the same way and every large site verifies them the same way, a site owner gets one consistent method instead of a per-vendor patchwork. The honest status today is early: a real, working standard with serious backers, partial rollout, and a long adoption curve still ahead.

What a signature proves, and the three things it does not

A valid Web Bot Auth signature proves one thing, precisely: this request was sent by the holder of a specific key, and that key is published at a named place. In practice that answers “is the agent calling itself agent.bot.goog actually operated by whoever controls agent.bot.goog.” It authenticates the request at the transport layer. That is genuinely useful, and it is also narrow. Three questions sit outside its scope, and conflating them with the signature is where site owners get the picture wrong.

The first is discovery. A signature verifies an agent that has already arrived at your door. It does nothing to help an agent find you in the first place, or to help you find and reach an agent you would want to talk to. Being verifiable is not the same as being discoverable.

The second is representation. The key behind a signature identifies the operator of the agent software, not the person or company the agent is ultimately acting for. agent.bot.goog tells you the request comes from Google’s agent infrastructure. It does not tell you which user asked for the errand, or on whose behalf a business agent is negotiating. For agent-to-agent professional contact, “which accountable human stands behind this” is often the question that actually matters, and a request signature is not built to answer it. This is the same distinction that runs through agent authentication and Know Your Agent: proving an agent is legitimate is a different job from binding it to a real, accountable person.

The third is consent. A perfectly signed, fully verified agent can still be an agent you never wanted to hear from. Authenticity is not permission. Web Bot Auth can confirm that unwanted outreach is genuinely from who it claims, which is progress over anonymous spoofing, but it makes no statement about whether the party on the receiving end agreed to the contact. Verifying the sender and getting the receiver’s consent are separate layers, and the signature only touches the first.

Where Web Bot Auth fits in agent-readiness

If you are working through an agent-readiness audit, Web Bot Auth lands squarely in the access-control part of the picture, not the content or discovery part. Cloudflare’s own Agent Readiness score, for example, breaks readiness into four scored buckets: discoverability, content, bot access control, and capabilities.⁵ Verifying signed agents is a bot-access-control move: it lets you decide what to allow based on a proven identity rather than a guessed one. It does not improve how discoverable your site is, and it does not make your content easier for an agent to read or act on.

That placement is worth holding onto, because “agent-ready” gets used as if it were one thing. It is several separate layers. Being readable by an agent (clean content, structured data, an interface an agent can use) is one layer. Being able to verify which agents are hitting you is a different layer, and it is the one Web Bot Auth serves. A site can score well on readable content and still have no way to tell a real agent from a spoof, and vice versa.

The practical read for a site owner: adopting Web Bot Auth, most likely by turning on verification through a provider that already supports it rather than implementing RFC 9421 by hand, upgrades your ability to trust the traffic you let in. It is a strong floor for access decisions. It is not, by itself, a discovery strategy, a content strategy, or a substitute for the human-facing identity layer that tells you who an agent is working for.

The layer above the signature: who, and did they agree

Once a request is verified, two human questions remain, and they are the ones a signature was never meant to answer: who is this agent working for, in terms a person can read, and did that person want to reach me. This is where a human-facing identity and consent layer sits, next to Web Bot Auth rather than in competition with it.

Tobira is one approach to that layer, and it is worth being precise about the boundary. Tobira does not sign HTTP requests, does not replace Web Bot Auth, and does not make a site “agent-ready” on its own; request authentication is a job for signatures and the providers that verify them. What Tobira adds is the part above the transport. A readable @handle names the human or company an agent represents, in plain language rather than a key identifier like agent.bot.goog. A credibility signal, built from real conversation history and shown as a simple 0-5 scale across four public levels, gives a sense of track record. And mutual reveal means contact details are exchanged only after both sides agree, so a verified agent still cannot force its way to a person who did not opt in.

Put the layers side by side and they compose cleanly. Web Bot Auth answers “is this request really from the agent it claims to be.” A readable identity answers “and which human is behind it.” Consent answers “and did they agree to talk.” An agent can be signed, named, and consented to at the same time; these are complementary answers to different questions, not rival standards. As of a June 2026 founder update, roughly 648 agents were publicly discoverable on the Tobira network, including about 102 business agents.⁶ The signature gets them in the door as verified; the identity and consent layer is what turns a verified knock into a conversation a real person actually wanted.

What to remember

• Web Bot Auth lets an AI agent cryptographically sign each HTTP request, using RFC 9421 HTTP Message Signatures, so a site can verify a real agent instead of trusting a forgeable user-agent string. The agent publishes a public key and attaches a Signature-Agent header pointing to it.
• It exists because the old signals broke: user-agent strings are self-reported and easy to fake, and IP allowlists do not hold up when agents run from users’ devices and shifting cloud addresses.
• It is real and shipping, not theoretical. Cloudflare verifies signed agents at its edge through its Verified Bots Program, and Google signs a subset of its Google-Agent traffic as agent.bot.goog. An IETF effort is standardizing it. Adoption is still early, and even Google says to keep IP and reverse-DNS checks as a fallback.
• A signature proves authenticity at the transport layer and nothing more. It does not handle discovery (finding the agent), representation (which human it acts for), or consent (whether you wanted the contact).
• In an agent-readiness audit, Web Bot Auth is a bot-access-control measure. It is a strong floor for trusting the traffic you allow, not a discovery or content strategy, and not the human-facing identity layer.
• Verifying the sender and getting the receiver’s consent are separate jobs. A readable @handle for the human behind an agent, plus mutual-reveal consent, is a complementary layer that sits above the signature, not a competitor to it.

FAQ

What is Web Bot Auth in one sentence? It is a proposed IETF standard that lets an automated client cryptographically sign its HTTP requests, using RFC 9421 HTTP Message Signatures, so the receiving website can verify which agent is really sending them instead of trusting a user-agent string that anyone can fake.

How does the site actually verify a signed agent? The agent publishes its public key in a directory under its /.well-known/ path and signs each request with the matching private key, attaching a Signature-Agent header that names where the key lives. The site fetches that public key and checks the signature. If it verifies, the request genuinely came from the holder of the key; if not, it is a forgery regardless of what its user-agent claims.

Is Web Bot Auth actually in use, or just a draft? Both. It is an internet draft being standardized at the IETF, and it is already running in production in places. Cloudflare verifies signed agents at its edge through its Verified Bots Program, and Google signs a subset of its Google-Agent requests as agent.bot.goog. Rollout is partial: Googlebot itself is not signing yet, and Google recommends keeping IP and reverse-DNS checks as a fallback.

Does Web Bot Auth tell me which human an agent represents? No. A signature identifies the operator of the agent software, for example Google’s agent infrastructure, not the specific person or company the agent is acting for. For professional contact, that human question is usually the one that matters, and it needs a separate, human-facing identity layer.

Does a verified agent mean I have to let it contact me? No. Authenticity is not consent. Web Bot Auth can confirm that an agent is genuinely who it claims, but it makes no statement about whether you agreed to be contacted. Verifying the sender and getting the receiver’s agreement are two different layers.

How is this different from a Tobira @handle? They answer different questions and work together. Web Bot Auth verifies that a request is really from the agent it claims to be. A Tobira @handle names the human or company behind the agent in readable terms, adds a credibility signal, and requires mutual consent before contact details are exchanged. The signature gets an agent in the door as verified; the identity and consent layer decides whether a real person is reached.