Sites whose robots.txt on the most recent scan explicitly disallowed at least one well-known AI agent (GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, and others). This isn’t a shame list: many sites block agents intentionally. The leaderboard simply records what we detected on the public /robots.txt at scan time. To remove your site from this list, allow the agents in question or remove the relevant Disallow rules.
Each row is the latest completed public scan for one domain. Composite is a 0–100 weighted average of five sub-scores; grade is bound to a published band table. The rubric (current and prior versions) is at /rubric.
No. We honour every site's robots.txt for our own scanner (AgentSpeedBot). This page reports what was found in their robots.txt for other agents, not whether we ourselves were allowed in.
Every score is the output of the public AgentSpeed rubric. The full check list, weights, and grade bands are at /rubric. New scans use the current active rubric; historical scans render against their original rubric.
Visit the scanner and submit a URL to generate a score. Every completed public scan creates a permanent score page and makes the domain eligible for ranking. There’s no application process and no editorial gate.
Add User-agent: AgentSpeedBot / Disallow: / to your robots.txt. On the next scan we honour it automatically and the public score page is replaced with a removal notice. See /bot.
Scan any public URL for a free agent-readiness score.
Side-by-side composite, sub-scores, and per-category winner. Permanent shareable URL.
Every check in the rubric has a remediation page with paste-ready snippets.
Crawler matrix, llms.txt analyzer, structured-data validator, and seven more.