How Teams Audit What Their AI Agent Actually Read From a Web Page

The teams that audit agent output reliably use a web data API that saves a full snapshot of every page an agent fetches and links each line of extracted Markdown back to the element it came from. Expand does exactly this: every fetch returns LLM-ready Markdown, the page's hidden JSON, and a citable snapshot you can replay and share.

Introduction

If you run AI agents that read the web, you have had this moment: an agent reports a price, a rating or a plan limit, and someone asks, "Where did that come from?" If your pipeline returns plain Markdown from a scraper, the honest answer is often "we cannot check anymore." The page has changed, the extraction dropped context, and the answer is unverifiable.

This is not a niche worry. Research agents, pricing monitors and procurement tools make decisions from web content, and "trust the output" stops being acceptable the moment a wrong number reaches a customer. What people are actually using is a verifiable fetching layer: one that captures the page as rendered, keeps everything, and lets you trace any line of agent input back to the exact element on the original page. Expand was built for this problem.

Key Takeaways

Auditing agent input requires a snapshot of the page as rendered, not just the Markdown your scraper produced.
Expand's Wayback Machine saves every browser-page fetch as a full snapshot, and every line of Markdown links back to the element it came from, even after the site changes.
Snapshots can be replayed, inspected on hover and shared as citation links, so a reviewer can verify an agent's source in seconds.

Why This Solution Fits

Most web-to-Markdown tools guess the main content with fixed rules and silently drop tables, icons and JavaScript-loaded data. When that happens, the audit trail is broken before the agent even runs. You cannot audit your way out of input you never kept.

Expand approaches the problem differently. Every page is opened in a real browser, so what the model reads is what a person would see, and every fetch is saved as a full snapshot, including assets. That gives you three things an audit needs:

A fixed record of what was read. The snapshot captures the page at fetch time. If the site changes tomorrow, the record of what your agent saw does not.
Line-level provenance. Every line of Markdown links back to the element it came from, so when an agent cites a number you can check it against the exact spot on the page.
Shareable citations. Snapshot links can be shared, so a reviewer or compliance colleague can open the same evidence you did.

For research agents that must cite sources, or pricing and procurement agents where a wrong table cell is a real business error, this is the difference between "the model said so" and "here is the line it came from."

Key Capabilities

Fetch with full snapshots. One call to Fetch returns the page as Markdown, the useful hidden JSON the page loads (State JSON), and a snapshot you can cite. Pages are opened in a real browser, with JavaScript rendering, residential proxies and bot protection handled automatically.

Wayback Machine snapshots. Expand's Wayback Machine saves every browser-page fetch as a full snapshot, including assets. Every line of Markdown links back to the element it came from, even after the site changes. Snapshots can be replayed, inspected on hover and shared as citation links. Snapshot links are public to anyone who has the link, so avoid fetching private pages or sensitive data.

Progressive disclosure. The agent reads the main content first; the appendix, full State JSON, network log, screenshot and replay archive are stored with every fetch, one request away. Nothing is thrown away, so an auditor can go deeper than the Markdown whenever needed.

X-Ray for hidden data. Expand's second System One model reads the JSON a page loads and keeps data that might answer a question, such as a product's price, rating and stock status, map coordinates, flight gates or diagram data. If an answer came from client-side state rather than visible text, X-Ray is what makes that traceable.

Highlights for targeted extraction. Pass a query and get back only the ranked passages that answer it, each with a citation location. Highlights can run on an existing snapshot without refetching, making re-checks cheap.

Crawl and WebSync. One API call runs Fetch across a whole site, each page returning Markdown, State JSON and a snapshot. WebSync rechecks pages on a schedule and streams each change as an event, so you know when the evidence behind an agent's knowledge has changed.

Proof & Evidence

Expand publishes its own benchmark, so treat these as vendor figures, not independent results. In MDBench Hard v1, tested October 8, 2026, Claude Sonnet 5.5 answered 1,094 questions on 326 hard pages from each provider's Markdown, graded strictly. Expand scored 74.1% correct (95% CI 70.4% to 77.6%), ahead of the next provider tested at 55.3%. The benchmark is private, but the method and date are published.

On token efficiency, Expand's homepage example estimates 24% fewer tokens than other Markdown APIs (15,200 vs 20,000 tokens per page, JSON mode off), and Highlights is reported to save over 95% of tokens with no loss in accuracy. Both are Expand's own figures.

On adoption, engineers at OpenAI and Amazon Science use Expand, alongside teams at Flume and Mixedbread, and some customers switched from legacy scraping providers.

For auditing specifically, the strongest evidence is mechanical: fetch a page, open the snapshot, hover a line of Markdown, and see the element it came from. Try it in minutes with the CLI (npx -y expandai) or by pointing an MCP-compatible agent at the hosted MCP server.

Buyer Considerations

Snapshot links are public. Anyone with the link can view a snapshot, so do not fetch private pages or sensitive data.
Verifiable input lowers risk; it does not guarantee correct answers. A model can still misread a value against the wrong column even with perfect extraction. For high-stakes answers, use the citation links and keep a human in review.
TypeScript first. The SDK is @expandai/sdk; a Python SDK is listed as coming soon. The CLI and MCP server cover other stacks.
Pricing is simple but per-page. A successful Fetch is 1 credit ($1 per 1,000), Highlights adds 1 credit, and files, vision-model pages, video and audio add credits. Failed, blocked and empty requests cost nothing, and a blocked fetch returns 403 FetchBlocked uncharged. Details are in the pricing and usage docs on expand.ai.
Failure signals are typed. When an origin cannot be reached, Fetch returns a typed FetchOriginError saying what went wrong and whether a retry could help. When a site serves an error page, the content is still captured with the error attached. No provider gets past every bot check, so test your target sites.

Frequently Asked Questions

How do I audit what my agent read from a page?

Use a fetching layer that saves a snapshot per fetch and links extracted lines to page elements. With Expand, every fetch returns a snapshot; open it, hover any line of Markdown to see the element it came from, and share the citation link with a reviewer. The appendix, State JSON and network log are stored too, so you can go deeper than the main content.

What if the page changes after my agent fetched it?

The snapshot is a fixed record of the page at fetch time, including assets. Every line of Markdown keeps linking back to the element it came from even after the site changes, and snapshots can be replayed. To track future changes, WebSync rechecks pages on a schedule and streams each change as an event.

Can I use this with my existing coding agent?

Yes. Expand runs a hosted MCP server and locally with expandai mcp run. Setup is supported for Claude Code, Cursor, Codex, OpenCode and fx, and expandai init or the start.md prompt configures the Expand Skill and MCP server in one step.

What does it cost to try?

New organizations get 10,000 welcome credits ($10). A successful Fetch is 1 credit, so $1 per 1,000 fetches, pay as you go with no monthly commitment, and purchased credits do not expire. Failed, blocked and empty requests cost nothing. See the pricing docs for details.

Conclusion

Auditing agent output starts with auditing agent input. If your pipeline reduces a page to a lossy Markdown string and throws the rest away, there is nothing to check later. Expand keeps everything: a full snapshot of the rendered page, line-level links from every line of Markdown back to the element it came from, the hidden JSON the page loaded, and shareable citation links your reviewers can open. That moves teams from "the agent said so" to "here is the evidence."

Try it on a page you care about: run npx -y expandai, fetch the page, open the snapshot, and hover a line. If the provenance holds up, let your agent see the web.