starlingbank.com

HTTP responses can carry a Link header: the same canonical and alternate-language information your HTML declares, but delivered in the response envelope itself. Agents that only send a HEAD request, or that decide what to do while the page is still streaming, read the header without parsing any HTML. Yours is missing (or carries none of the relations agents use: canonical, alternate, describedby).

An agent triaging fifty URLs doesn’t want to download and parse fifty pages to learn which are duplicates of which. The Link header answers at the cheapest possible layer. Sites that provide it get correctly de-duplicated and correctly language-routed even by the most minimal fetchers.

1. Emit a Link header alongside each page, mirroring what your HTML head already declares. Example: Link: <https://yoursite.com/page>; rel="canonical".
2. Most stacks set this in one place: Next.js headers() config, an nginx add_header line, or a CDN response-header rule.
3. If you serve translations, add rel="alternate" entries with hreflang, matching your HTML.

• Run: curl -sI https://yoursite.com | grep -i "^link:" and confirm the relations appear.
• Hit Re-scan above: "Link response headers" flips to PASS.

• The header must AGREE with the HTML. A Link header pointing one place while the in-page canonical points another gives machines two contradictory answers, which is worse than one missing answer.
• Don’t inject the same site-wide canonical on every page via a blanket CDN rule; each page names its own clean URL, exactly as in the HTML tag.

Of everything your server sends for this page, very little is actual readable text. The rest is code, styling, and markup wrapper. Agents fetched a lot of bytes and found few words.

Agents work with retrieval budgets. A page that is 2% text either gets skimmed (and mis-summarised) or skipped. More signal per byte means more of your actual message survives into the agent’s summary.

1. Confirm your real copy is server-rendered. See the JavaScript check; these two usually fail together.
2. Cut boilerplate wrappers: deeply nested divs, inline SVG logos repeated per section, base64 images inlined into HTML.
3. Move large inline styles and scripts into external files.
4. Check how much of the document is inline script: your report now says. If most of it is, that is your framework serializing state for hydration, often a second copy of text already in the HTML. Send only the props a component cannot recompute, and render statically where a page does not need to hydrate.

• Re-scan: "Text-to-markup ratio" improves, and "Page weight" often improves alongside.

• Don’t pad the page with keyword text to inflate the ratio. The ratio is a proxy for substance, and stuffing is the opposite of substance.

Your page’s actual content (headlines, product info, pricing) only appears after JavaScript runs in a browser. Most AI agents read the raw HTML that arrives FIRST, and for your site, that first delivery is nearly empty.

An agent fetching your page gets the skeleton: navigation links, a footer, loading placeholders. It then summarises your business from a skeleton. Sites in this state get described as "this page contains little information" while the full pitch renders only for humans.

1. Next.js: Render primary content on the server: use Server Components (App Router) or getServerSideProps/SSG (Pages Router). Avoid fetching above-the-fold content in a client useEffect, since agents don’t run it.
2. Server-render the content that matters: hero copy, value proposition, pricing, product details.
3. Next.js: use Server Components or static generation for marketing pages. Don’t fetch above-the-fold content in a client useEffect.
4. Quick self-test: view the page source (Ctrl+U shows the raw HTML, not the inspector) and search for your headline. If it isn’t there, agents don’t see it.

• Your headline and pricing appear in the raw page source.
• Re-scan: "Content without JavaScript" flips, and "Text-to-markup ratio" usually improves with it.
• The "What the agent receives" section below shows exactly what a no-JS agent gets from your page.

• Don’t serve a special pre-rendered version only to detected bots (cloaking). Agents compare notes with real user agents, and a divergence gets you classified as deceptive.
• Don’t paste an HTML copy of the content into a hidden div as a workaround. Duplicated content that drifts from the rendered version fails coherence in the ugliest way.

Headings (H1, H2, H3…) are the page’s outline. Yours skip levels, repeat H1s, or use styled text instead of real heading tags, so the outline machines extract is scrambled.

Agents use the heading tree to navigate long pages and to decide which section answers a question. A broken tree means quotes get attributed to the wrong section, or sections are missed entirely.

1. Exactly one H1 per page: the page’s actual title.
2. Nest without skipping: H2 for sections, H3 for subsections. Never jump H1 to H4 for visual size.
3. If a heading looks right but is a styled <div>, make it a real <h2> and adjust CSS. Machines only see tags.

• Re-scan: "Heading hierarchy" flips to PASS.

• Don’t pick heading levels for their font size; style with CSS and keep levels semantic.

Time to first byte is how long your server thinks before it starts answering: the gap between an agent asking for the page and the first byte of HTML arriving. Yours crossed our threshold (under 200ms scores full marks, over 2 seconds fails).

Agents fetch with tight timeouts and many pages in parallel. A server that takes seconds to start answering gets cut off mid-queue, and a page that times out is a page that was never read. Speed here is not polish; it decides whether the content gets fetched at all.

1. Cache the HTML itself. Marketing and docs pages rarely change per visitor; serve them from a CDN or static cache so no application code runs per request.
2. If you are on serverless hosting, cold starts are a common cause: keep the function warm or pre-render the page.
3. Move slow work (database queries, third-party API calls) out of the first response; render the shell fast and fill personalised details after.

• Run: curl -o /dev/null -s -w "%{time_starttransfer}\n" https://yoursite.com and watch the number drop.
• Hit Re-scan above: "Time to first byte" improves a bucket at a time (under 500ms passes).

• Don’t answer fast with an empty shell that loads content by JavaScript. That trades this warning for the much worse "content requires JavaScript" failure, since most agents never run your scripts.
• Don’t special-case known crawler user agents onto a faster path. Serving different performance to different audiences is the pattern crawlers penalise when they detect it.