Detected restrictions on AI agents
Sites whose robots.txt on the most recent scan explicitly disallowed at least one well-known AI agent (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, and others). This isn’t a shame list: many sites block agents intentionally. The leaderboard simply records what we detected on the public /robots.txt at scan time. To remove your site from this list, allow the agents in question or remove the relevant Disallow rules.
Not enough scan data yet.
We surface only real, completed public scans. Once enough domains with detected restrictions have a public score on AgentSpeed, this leaderboard fills in automatically.
Scan your site →Each row is the latest completed public scan for one domain. Composite is a 0–100 weighted average of five sub-scores; grade is bound to a published band table. The rubric (current and prior versions) is at /rubric.
Does AgentSpeed blocklist these sites?
No. We honour every site's robots.txt for our own scanner (AgentSpeedBot). This page reports what was found in their robots.txt for other agents, not whether we ourselves were allowed in.
How are scores calculated?
Every score is the output of the public AgentSpeed rubric. The full check list, weights, and grade bands are at /rubric. New scans use the current active rubric; historical scans render against their original rubric.
How do I get my domain on the leaderboard?
Visit the scanner and submit a URL to generate a score. Every completed public scan creates a permanent score page and makes the domain eligible for ranking. There’s no application process and no editorial gate.
How do I remove my site?
Add User-agent: AgentSpeedBot / Disallow: / to your robots.txt. On the next scan we honour it automatically and the public score page is replaced with a removal notice. See /bot.
Scan any public URL for a free agent-readiness score.
Side-by-side composite, sub-scores, and per-category winner. Permanent shareable URL.
Every check in the rubric has a remediation page with paste-ready snippets.
Crawler matrix, llms.txt analyzer, structured-data validator, and seven more.