Remediation
robots.txt advertises Content-Signal directives (e.g. `Content-Signal: ai-train=no, ai-summarize=yes`) that let agents respect granular usage preferences.
robots_txt.content_signalsdiscoverabilityweight 1deterministic
Why it matters
robots.txt advertises Content-Signal directives (e.g. `Content-Signal: ai-train=no, ai-summarize=yes`) that let agents respect granular usage preferences.
Underlying specification: https://contentsignals.org
How to fix
- See the full check definition in the rubric (linked above) for the canonical reference.
- Read the underlying specification at https://contentsignals.org.
Hand it to your AI assistant
Paste this into Claude Code, Cursor, or any assistant with your repository open. It carries the steps, how to verify them, and the shortcuts to avoid. It is generic, though: run a scan to get one written from what your pages actually do.
Run a scan to check your site
Verify on your site
Run a scan against your domain to see whether robots_txt.content_signals currently passes, warns, or fails. Free, no signup; the score page is permanent and citable.