agentspeed.

unstructured.io

scanned 7/25/2026, 11:40:31 PM · cached · refresh in 23h · rubric 2026.07.4
Built with Webflow
robots.txt present: failrobots.txt allows AI agents: passContent-Signal directives: failllms.txt present: warnSitemap present: warnLink response headers: failCanonical URL: passMCP server card: failOAuth authorization metadata: failOAuth resource metadata: failAPI catalog / OpenAPI: failWeb Bot Auth: failMarkdown negotiation: failText-to-markup ratio: failContent without JavaScript: skipHeading hierarchy: failCookie wall blocks content: warnPage title: passMeta description: passJSON-LD present: passStructured data validates: warnSchema type coverage: passPaywall / login wall: passAgent Skills manifest: failWebMCP actions: failPrimary action reachable: passReachability: passTime to first byte: passFull render time: skipPage weight: warn68/ 100 · D
Needs work

This is how AI agents (not browsers) experience unstructured.io. The score weights five categories of machine-readability; the ticks on the arc are the 30 individual checks behind it.

Discoverability
41
Readability
66
Structured data
98
Actionability
75
Performance
73

✉ Email me this report + alert me when it changes ↓

Top fixes

failrobots.txt presentDiscoverability

No reachable robots.txt at the site root. Publish one so agents can discover your crawl policy and sitemap.

Fix: Add a `/robots.txt` file at the site root. Even an allow-all file signals intent to agents.

warnllms.txt presentDiscoverability

Found /llms.txt but missing H1 header and markdown links.

Fix · Webflow: Publish /llms.txt at your domain root. Build a spec-compliant one at /tools/llms-txt-generator.

warnSitemap presentDiscoverability

sitemap.xml reachable but not referenced from robots.txt, so agents may miss it.

Fix · Webflow: Publish /sitemap.xml and reference it from robots.txt with a Sitemap: line.

failLink response headersDiscoverability

No HTTP Link: response headers. Emit canonical/alternate/describedby relations so HEAD-only or stream-rendering agents get them without parsing HTML.

Fix: Emit `Link:` headers for canonical, alternate-language, and describedby relations. Agents that fetch HEAD-only or stream-render rely on these.

failText-to-markup ratioReadability

Visible text is 0.2% of HTML weight (4503/2209407 bytes). Low. Markup may be crowding out agent-usable text.

Fix: Heavy ad or navigation markup crowds out agent-usable text. Aim for a text-to-DOM ratio above 15%.

failHeading hierarchyReadability

Heading issues: 10 <h1> tags (expected 1); 5 skipped level(s).

Fix: Use a single H1 and avoid skipping heading levels (H1 → H3).

Don’t lose this report

Get it in your inbox now — and a heads-up whenever unstructured.io’s agent-readiness score changes. Free, unsubscribe anytime.

Fix it

Generated, ready-to-ship files for the gaps above — copy them or download and drop them into your repo.

Add an llms.txt
Publish this at https://unstructured.io/llms.txt
# Unstructured Data Platform for GenAI | Unstructured

> Transform complex, unstructured data into clean, AI-ready inputs. Connect to any source, process 64+ file types, and power your GenAI projects. Start now.

## Key pages
- [Home](https://unstructured.io/)
- [Whatever It Is, We Can Structure It.](https://unstructured.io/)
- [Trusted by oo 87 % of the Fortune 1000.](https://unstructured.io/)
- [ETL plus plus so much more.](https://unstructured.io/)
- [Transform over iO 64 different file types.](https://unstructured.io/)
- [Industry-awarded. Enterprise-trusted.](https://unstructured.io/)
- [Built to connect. Designed to scale.](https://unstructured.io/)

## About
Describe what unstructured.io does, who it's for, and the primary action you want an
agent to be able to complete. Keep it factual and current.

What the agent receives

The agent’s-eye view
Unstructured Data Platform for GenAI | Unstructured Your Cart Nothing here yet Just like good ideas, great swag starts with a spark. Browse around and make it yours. Check our Swag Store Product Foundation Problems We Solve Resources Pricing Partners Login Book A Demo Get Started for Free We&#x27;re Back on Fast Company&#x27;s Most Innovative list We&#x27;re Back on Fast Company&#x27;s Most Innovative list Whatever It Is, We Can Structure It. Transform complex, unstructured data into clean, structured data. Securely. Continuously. Effortlessly. Book A Demo Trusted by oo 87 % of the Fortune 1000. We Orchestrate, You Innovate ETL plus plus so much more. Security and compliance? Built in. Role-based access? Handled. We take care of all the things that slow teams down so you can focus on unlocking the full potential of your data. Extract Transform Parse, chunk, embed, and enrich data. 64+ File Types Chunking, Enrichment, Embedding Open AI, Anthropic, + more Integrations With fast speeds, and seamless partner integrations, our transformation is optimized for any destination. Learn More Load Plus + Extract Transform Parse, chunk, embed, and enrich data. Load Plus + 64+ File Types Chunking, Enrichment, Embedding Open AI, Anthropic, + more Integrations With fast speeds, and seamless partner integrations, our transformation is optimized for any destination. Learn More Drag. Drop. Done. Transform over iO 64 different file types. Grab one of the files below and watch Unstructured turn messy data into clean, structured output, ready for AI and analysis. Invoice.pdf Database.csv Newsletter.pdf Invoice.pdf Try Unstructured Drop a file here Industry-awarded. Enterprise-trusted. CB Insights Top 100 AI Companies “The AI 100 winners are future movers and shakers, taking big swings that will shape the future of AI.” CB Insights Forbes Top 50 AI Companies “Unstructured is part of a new wave of AI startups completing real workflows.” Forbes Fast Company #24 Most Innovative “Unstructured helps companies unlock the AI potential of their raw data.” Fast Company Gartner Gartner Cool Vendor 2024 &quot;Recognized for innovation, impact, and the ability to solve enterprise challenges in novel ways.&quot; Gartner Every Source &amp; Destination Built to connect. Designed to scale. With 30+ connectors and 1,250+ pipelines, we seamlessly integrate with any database, data lake, or enterprise system. And because connections should, you know, stay connected, our 24/7 maintenance ensures your pipelines stay reliable as systems evolve. View Connectors Avoid The Rat’s Nest Just because you can build it, doesn&#x27;t mean you should. Building your own document processing pipeline starts simple—but scaling it is another story. What begins as a few scripts and connectors quickly turns into a tangled mess of maintenance, custom fixes, and never-ending updates. Unstructured replaces the DIY rat’s nest so you can focus on AI innovations that move your business forward. Avoid The Rat’s Nest UI &amp; MCP Interface options for everyone. Rather click through it yourself? There’s a UI for that. Rather your agent just handle it? There’s an MCP for that. UI Our UI makes it easy for teams to process and transform data without heavy coding. Explore UI MCP Document parsing built for agents. One call in Claude Code, Cursor, or Codex, and structured data lands in context. Explore MCP Your Database, Our Pre-Processing Data delivered to your doorstep. Learn More Learn More Learn More Learn More Learn More Learn More Learn More Learn More If you’re already storing your data with one of our trusted partners, integrating Unstructured into your preprocessing workflow is effortless. Get started with one of our partner setup guides and you&#x27;ll be up and running in no time. View Partners Try It Out Now Ready for a demo? See how Unstructured simplifies data workflows, reduces engineering effort, and scales effortlessly. Get a live demo today. Contact Sales Simplify Workflows Designed To Scale Built To Connect Visit Unstructured’s Trust Portal to learn more. Quick Links Homepage Product Foundation Problems We Solve Pricing Resources Benchmarks Careers Blog Webinars &amp; Events Press Partners Insights Tutorials Government Docs Whatever it is, we can structure it. Join our newsletter. Subscribe Contact Us Ready to get started? Privacy Policy Cookie Notice Terms &amp; Conditions Your Privacy Choices Copyright © 2026 Unstructured 1 2 3 4 5 6 7 8 9 10 11 12

Show your score

Embed this badge on your site. It links back to this live scan and updates on every rescan.

AgentSpeed score for unstructured.io
<a href="https://agentspeed.com/scan/unstructured.io"><img src="https://agentspeed.com/badge/unstructured.io" alt="AgentSpeed agent-readiness score" height="56"></a>

Track this score

Get this report by email, then a heads-up when unstructured.io’s agent-readiness actually changes — a check regressing, or the composite dropping.

Two things never trigger an email: a score change caused by us publishing a new rubric, because that is a different instrument rather than a change to your site, and movement explained only by timing-derived checks, which shift run to run on a site nobody touched.

Want it on your own schedule, with Slack or a webhook instead of email? Score-drift monitoring is on every paid plan.

Full breakdown

10 pass5 warn13 fail2 skip
Discoverability
fail
robots.txt present
No reachable robots.txt at the site root. Publish one so agents can discover your crawl policy and sitemap.
0/100
pass
robots.txt allows AI agents
All 8 answer-time access agents allowed.
100/100
fail
Content-Signal directivesemerging
No Content-Signal directives. Add e.g. `Content-Signal: ai-train=no, ai-summarize=yes` to declare granular AI usage policy beyond binary allow/disallow.
0/100
warn
llms.txt present
Found /llms.txt but missing H1 header and markdown links.
60/100
warn
Sitemap present
sitemap.xml reachable but not referenced from robots.txt, so agents may miss it.
70/100
fail
Link response headers
No HTTP Link: response headers. Emit canonical/alternate/describedby relations so HEAD-only or stream-rendering agents get them without parsing HTML.
0/100
pass
Canonical URL
Canonical points to self: https://unstructured.io/.
100/100
fail
MCP server cardemerging
No valid MCP Server Card at /.well-known/mcp/server-card.json. Publish one so agents can discover your tools without HTML scraping.
0/100
fail
OAuth authorization metadataemerging
No RFC 8414 metadata at /.well-known/oauth-authorization-server. Agents that act on behalf of users need this to discover your authorization endpoints.
0/100
fail
OAuth resource metadataemerging
No RFC 9728 metadata at /.well-known/oauth-protected-resource. Publish it so agents can discover required scopes without hand-coded credentials.
0/100
fail
API catalog / OpenAPIemerging
No /.well-known/api-catalog (RFC 9727) and no /openapi.json|yaml. Publish one so agents that integrate with APIs can discover your endpoints.
0/100
fail
Web Bot Authemerging
No Web Bot Auth JWKS at /.well-known/http-message-signatures-directory.json. Publish one to allow trusted agents while keeping a default-deny posture for the rest.
0/100
Readability
fail
Markdown negotiationemerging
Accept: text/markdown returns HTML, not markdown. Serve a markdown variant of primary content when requested; agents summarize and cite it more reliably.
0/100
fail
Text-to-markup ratio
Visible text is 0.2% of HTML weight (4503/2209407 bytes). Low. Markup may be crowding out agent-usable text.
2/100
skip
Content without JavaScript
Browser pass unavailable; cannot measure JS dependency.
—
fail
Heading hierarchy
Heading issues: 10 <h1> tags (expected 1); 5 skipped level(s).
50/100
warn
Cookie wall blocks content
Consent modal detected (cookielaw.org); content still visible. Verify the modal does not block first-paint for screenshot agents.
70/100
pass
Page title
A concise <title> is present (51 chars).
100/100
pass
Meta description
A well-sized meta description is present (154 chars).
100/100
Structured data
pass
JSON-LD present
Found 1 JSON-LD block.
100/100
warn
Structured data validates
1 recommended-field warning(s). First: WebSite recommends "potentialAction" (improves agent comprehension). 1 block(s) skipped — type outside the set we check.
95/100
pass
Schema type coverage
Page intent unclear; no specific schema.org type expected.
100/100
Actionability
pass
Paywall / login wall
No paywall or login-wall detected on the landing URL.
100/100
fail
Agent Skills manifestemerging
No Agent Skills manifest at /.well-known/agent-skills.json. Enumerate the tasks agents can perform (search, add-to-cart, contact-support) so they pick the right one without scraping.
0/100
fail
WebMCP actionsemerging
No WebMCP detected. On pages with first-class actions (cart, support, account), embed a WebMCP server so on-page agents invoke tools directly.
0/100
pass
Primary action reachable
Primary offering identified ("Unstructured Data Platform for GenAI") with price/CTA ("Get Started for Free").
100/100
pass
Reachability
HTTP 200.
100/100
Performance
pass
Time to first byte
TTFB 328ms. Healthy.
80/100
skip
Full render time
Full-render time unavailable (browser pass skipped or failed).
—
warn
Page weight
Initial document is 2158 KB. Under 3 MB, but trimming toward ~500 KB keeps agent fetches fast.
60/100
Beyond the scan

Running AI agents of your own? AgentSpeed also monitors them in production — every run, its latency, cost, and failures, with alerts when something breaks. Free for 10,000 events a month, no credit card.

Start watching free →See plans