Remediation

Failing: robots_txt.allows_known_agents

robots_txt.allows_known_agentscriticalauto-fix available

Generated fix

Generated fixreplace file→ https://lrt.lt/robots.txt

Your current /robots.txt disallows these AI agents: ChatGPT-User, OAI-SearchBot, PerplexityBot, Amazonbot. Replace your robots.txt with the version below to explicitly allow them while keeping any custom rules you've added below the wildcard block.

# robots.txt for lrt.lt
# Explicitly allow well-known AI agents that were previously blocked.
# Adjust or remove specific agents as your policy requires.

User-agent: *
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Amazonbot
Allow: /

Sitemap: https://lrt.lt/sitemap.xml

save as https://lrt.lt/robots.txt

›This snippet replaces your existing robots.txt. Merge any custom Disallow rules you have today below the wildcard block.
›Cloudflare and a few CDNs cache robots.txt aggressively, so purge the cache after deploy to confirm AI agents see the change.

Why and how

4 access agent(s) blocked: ChatGPT-User, OAI-SearchBot, PerplexityBot, Amazonbot. These keep your site out of AI answers. 7 training-data crawler(s) blocked (GPTBot, ClaudeBot, anthropic-ai, Google-Extended, Bytespider, CCBot, Meta-ExternalAgent), a licensing choice, not scored.