zhihu.com

Before reading anything, an agent has to reach your page: one request, a clean answer. Yours didn’t answer cleanly. The finding above says exactly what happened: an error status, a connection failure, or a chain of redirects before any content arrived.

Agents give up much faster than browsers. A human waits through three redirects and a retry; an agent on a time budget records "unreachable" and answers from other sources. Every other measurement on this page happens only after this one succeeds, so this failure hides everything else your site does right.

1. Read the finding above: it names the status code and counts the redirects we followed.
2. If it is a redirect chain, collapse it to one hop. The classic chain is http → https → www → final page; configure your host to jump straight from any variant to the final address in a single redirect.
3. If it is a 4xx or 5xx status, your server is refusing or failing the page. Fix the route or the error before anything else on this page.
4. If the fetch failed outright, check DNS and the TLS certificate; an expired or mismatched certificate turns every agent away at the door.

• Run: curl -I https://yoursite.com and confirm a 200, or a single 301 followed by a 200.
• Hit Re-scan above: "Reachability" flips to PASS, and checks that were skipped get measured for the first time.

• Don’t serve an error page with a 200 status to look reachable. Agents then read the error text as your content and summarise your business from it.
• Don’t block by IP reputation so aggressively that assistant crawlers are refused while browsers pass; the finding above reflects what an agent gets, not what you see.

Content-Signal is a newer robots.txt directive (pushed by Cloudflare) that lets you state a granular AI policy: for example "don’t train on my content, but summarising it in answers is fine". Classic robots.txt only offers all-or-nothing per crawler; this adds the middle ground. We looked for a Content-Signal line in your robots.txt and response headers and found none.

Without a granular signal, crawlers infer your intent from blunt allow/block rules, and sites often block more than they mean to just to avoid training use. A declared signal lets you keep answer visibility (summaries, citations) while opting out of what you object to. This is an emerging standard: honoring is voluntary and adoption is early, which is why it is flagged as emerging and weighs little.

1. Decide your actual policy first: are you fine with AI training on your content? With summarisation in answers? With search indexing?
2. Express it as one line in robots.txt, for example: Content-Signal: ai-train=no, ai-summarize=yes.
3. Keep your existing User-agent rules; the signal adds nuance on top, it doesn’t replace them.

• Open yoursite.com/robots.txt and confirm the Content-Signal line is live.
• Hit Re-scan above: "Content-Signal directives" flips to PASS.

• Don’t declare signals that contradict your User-agent rules, such as ai-summarize=yes while blocking every AI crawler; conflicting instructions get you treated as unreliable.
• Don’t add the line just for the score. It is a public policy statement; say what you mean, because compliant crawlers will act on it.