agentspeed.
← all tools
Tool

Agent UA tester: see your site through five AI crawlers

Many sites cloak: serve a static shell to bots, real content to humans. Agents see the shell. Result: hallucinated answers about your product. We fetch your URL six times in parallel, once each as AgentSpeedBot, GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, and a vanilla desktop Chrome, then diff the responses.

Tool
›

Agents we send

AgentSpeedBot
Our scanner (baseline)
AgentSpeedBot/1.0 (+https://agentspeed.com/bot)
GPTBot
OpenAI training crawler
Mozilla/5.0 (compatible; GPTBot/1.2; +https://openai.com/gptbot)
ChatGPT-User
ChatGPT browsing on behalf of a user
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
PerplexityBot
Perplexity citation crawler
Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
ClaudeBot
Anthropic crawler
Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)
Browser
Vanilla desktop Chrome (control)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36
Examples

Click an example to run the tool with pre-filled inputs.

Why this matters for AI agents

Agents that get a different response than your real users misrepresent you. A 403 means the agent has nothing to cite. A redirect to /login means agents see only your auth wall. A static shell means agents read your nav menu and footer instead of your product copy.

AgentSpeed runs an end-to-end version of this diff as part of the journey.primary_probe check on every public scan.

Common fixes
AI bots redirected to /login
Render server-side preview content for unauthenticated requests on key landing pages: pricing, docs, marketing. Agents need a cite-able shell, not the auth wall.
Cloudflare challenges block GPTBot
In Cloudflare bot management, allow GPTBot, ClaudeBot, and PerplexityBot explicitly. Their UA strings are documented and the IPs are fixed.
Static shell with no copy
Server-side render the hero copy, value props, and pricing. JS-only sites force agents to read a near-empty body and infer everything from the URL.
Different titles per UA
If <title> differs across agents, your CMS or edge layer is rewriting metadata for bots. Audit any "agent-aware" rendering rules, since agents often get the worst variant.
Canonical URL drift
Canonicals should not differ between UAs. If your site adds tracking parameters per-bot, agents will see splintered URLs and dedupe poorly.
Linked rubric checks
FAQ
Is cloaking against Google’s policy?
Serving different content to Googlebot than to human users violates Google’s Webmaster Guidelines. Serving different content to AI agents is not formally policed by Google but tends to produce worse outcomes, because agents that get a static shell hallucinate answers based on partial information.
Why does my site differ for GPTBot?
Common causes: bot management products (Cloudflare, Akamai) issuing challenges; CMS rules that strip personalisation for bots; CDN routes that serve a marketing variant. Test in this tool to see exactly which agents get which variant.
What is a normal similarity score?
Above 0.95 is essentially identical. Below 0.80 means meaningful drift. A score under 0.50 between bots and the browser usually means cloaking or a content firewall.
Will fixing cloaking improve agent answers?
Yes. Agents that get richer body content cite more accurately. Sites that ship the same SSR shell to bots and users tend to outperform in agent-cited answers.
Do you store the responses?
No. Each run is fetched live. Nothing is persisted. The result page is rendered server-side and discarded.
Run a full agent-readiness scan
Full scan

Score the entire site against 30 agent-readiness checks across discoverability, readability, structured data, actionability, and performance.

Get the full report →
Agent UA tester: see your site through five AI crawlers · AgentSpeed