agentspeed.
Rubric / Discoverability

Content-Signal directives

robots_txt.content_signals
weight 1deterministicemerging

robots.txt advertises Content-Signal directives (e.g. `Content-Signal: ai-train=no, ai-summarize=yes`) that let agents respect granular usage preferences.

Adoption of what this check tests is still climbing, so failing it is an opportunity rather than a defect. Remediation lists rank these after established fixes, and the weight moves only when measurement earns it.

Definition
Check ID
robots_txt.content_signals
Category
Discoverability
Weight
1
Counts toward the score
Yes.
Evaluated by
A fixed rule with no threshold of our choosing. The same input always produces the same verdict.
Standard maturity
emerging
Canonical spec
https://contentsignals.org
Introduced in
r2026.04.2
Retired in
Still active.
History
Every change to the instrument— including the versions that changed this check
Machine-readable
/rubric.json— this check, and every other, as data
How often this check is right

No rate published. Too few labelled cases to publish a rate: 1 flagged case (30 needed to publish), 1 defective case (30 needed to publish). The counts are the honest answer here. The matrix behind robots_txt.content_signals is 2 labelled cases — reported here rather than turned into a percentage that would read as more certain than the evidence is.

Corpus v1, measured under rubric r2026.11.0. Full method and every other check at /rubric/accuracy.

How to fix it
Add `Content-Signal:` directives to your robots.txt. This is the emerging standard (Cloudflare-driven) for declaring fine-grained AI usage policy beyond a binary allow/disallow.
Does your site pass this check?

Free scan, 30 seconds, no signup — this check and the other 35, with the exact evidence for each verdict.

Scan my site →
Content-Signal directives · robots_txt.content_signals · AgentSpeed