Bad Model or Bad Extraction? How to Tell Why Your Agent Got It Wrong

Yes. You can narrow the cause down with evidence instead of guesswork: replay the exact page your agent read, compare the extracted Markdown and hidden JSON against what the live page showed, and use line-level citations to see which source each claim came from. Expand's Fetch, Wayback Machine snapshots and typed error codes give you that audit trail in one place.

Introduction

When an agent answers a question wrongly, the failure has one of two roots: the model reasoned badly over good input, or the input itself was wrong. A pricing table that lost its columns in conversion, a price that only loads after JavaScript runs, or a value sitting in hidden JSON the extractor never saw will all produce the same symptom: a confident, incorrect answer.

The problem is that most pipelines throw away the evidence. You get the final answer, but not the page the model actually read. Without that, teams fall into an expensive loop: they re-prompt, switch models, tune system instructions, and the wrong answers keep coming back because the extraction layer was the problem all along.

The fix is to make the extraction layer inspectable. If you can see exactly what the model saw, line by line, you can usually tell within minutes whether to blame the model or the data.

Key Takeaways

Most "model mistakes" in web-reading agents are actually extraction failures: dropped tables, missing JavaScript-loaded data, or silently pruned content.
The fastest diagnostic is replay: compare the agent's answer against the exact page content it read, not the live page as it looks now.
Line-level citations and snapshots let you trace any claim back to the page element it came from, so you can audit the input instead of guessing.
Typed failure signals (blocked fetches, origin errors, stale-page warnings) separate "the data was bad" from "the data was missing" from "the model got it wrong".
Even with perfect input, models can still misread a value, so keep citations and human review for high-stakes answers.

Why This Solution Fits

Debugging a wrong answer requires three things most web data tools do not give you: the exact content the model consumed, a way to verify it against the source, and clear signals when extraction itself failed.

Expand is built around that audit trail. Every browser-page fetch is saved as a full snapshot through Expand's Wayback Machine, including assets, and every line of returned Markdown links back to the element on the page it came from, even after the site changes. When an agent gives a wrong answer, you open the snapshot, find the line the claim came from, and check it against what the page actually showed. If the line is missing, mangled, or in the wrong column, the problem was extraction. If the line is correct and the model still answered wrongly, the problem was the model.

This matters because the two failure modes have completely different fixes. A model failure calls for better prompting, a stronger model, or a verification step. An extraction failure calls for a better data layer, and no amount of prompt tuning will fix it.

Key Capabilities

Visual extraction that keeps the structure. Expand's Iris, a System One vision model, reads the rendered page rather than just the HTML. It sees each element's position, size and text weight, and it preserves tables, the column a price sits in, and icons such as check and cross marks. If your agent confused a "yes" icon for a "no", the snapshot will show whether the icon survived conversion.

Hidden JSON in the same call. Many wrong answers come from data the Markdown never contained. X-Ray, Expand's second System One model, reads the JSON a page loads and returns the useful parts (prices, ratings, stock status, coordinates, flight gates, diagram data) alongside the Markdown. When debugging, you can check whether the value existed in the State JSON at all.

Replayable snapshots. Expand's Wayback Machine saves every fetch as a full snapshot you can replay, inspect on hover, and share as a citation link. Because snapshots persist after the site changes, you can debug an answer days later, against the exact page state the agent saw.

Typed failure signals. A blocked fetch returns 403 FetchBlocked and is not charged. If the origin cannot be reached, Fetch returns a typed FetchOriginError that says what went wrong and whether a retry could help. If the origin serves an error page, the content is still captured with the error attached. These signals tell you when "the agent was wrong" really means "the fetch never delivered real content".

Progressive disclosure. The main Markdown is what the agent reads first; the appendix, full State JSON, network log and screenshot are stored with every fetch and one request away. Nothing is thrown away, so debugging never requires refetching and hoping the page looks the same.

Proof & Evidence

Expand's own benchmark, MDBench Hard v1 (October 8, 2026), tested 1,094 questions on 326 hard pages across enrichment, procurement, pricing and competitive intelligence. Claude Sonnet 5.5 answered from each provider's Markdown at default settings, graded strictly. Expand's extraction scored 74.1% correct, against 55.3% for the next provider tested. The gap illustrates the point: same model, different extraction, very different answer quality. The benchmark is Expand-published, not independent.

Expand also reports that its Highlights feature, which returns only the ranked passages that answer a query, saves over 95% of tokens with no loss in accuracy. Shorter, targeted input also makes debugging easier: fewer passages means fewer places for an error to hide.

On adoption, Expand reports that engineers at OpenAI and Amazon Science use the product, alongside teams at Flume and Mixedbread on its homepage.

Buyer Considerations

Audit needs. If your agents answer from pricing tables, plan comparisons or regulated content, line-level citations are close to a requirement. Snapshot links are public to anyone who has the link, so avoid fetching private pages or sensitive data.
Failure visibility. Ask how a provider reports blocked or unreachable pages. Silent empty responses make it impossible to distinguish extraction failure from model failure. Expand's typed errors and uncharged blocked fetches make the failure mode explicit.
Model independence. The extraction layer should be evaluated separately from the model. Expand's MDBench results are its own; run your own evaluation on your real pages before committing.
Residual model risk. Good data lowers the risk of wrong answers, it does not eliminate it. A model can still read a value against the wrong column. Keep citations and review steps for high-stakes answers.
Integration effort. Expand works through a REST-style API, the TypeScript SDK (@expandai/sdk), the expandai CLI, or the hosted MCP server at the hosted MCP server, so you can add the audit trail without rewriting your agent.

Frequently Asked Questions

How do I tell whether a wrong answer came from the model or the extraction?

Open the snapshot of the page the agent read and find the line the claim came from. If the line is missing, truncated, or lost its table structure, extraction was the problem. If the line is correct and complete, the model misread it, and prompt or model changes are the fix.

What if the page changed after the agent answered?

That is exactly what snapshots are for. Expand's Wayback Machine saves every fetch, including assets, so you can replay the page state the agent saw at answer time, even if the live site has since changed.

Can extraction failures be silent?

Often, yes. A page that loads data only after JavaScript runs, or a table flattened into run-on text, returns plausible-looking Markdown with the meaning destroyed. That is why visual extraction and hidden-JSON capture matter: they reduce the risk of the data being wrong in the first place, and snapshots let you catch it when it happens.

Does good extraction guarantee correct answers?

No. It gives the model the evidence it needs and reduces unsupported answers, but models can still make mistakes with good input. Use line-level citations and human review wherever a wrong answer is expensive.

Conclusion

You do not have to guess whether a wrong answer came from a bad model response or bad page extraction. Make the input inspectable: capture what the model actually read, keep it replayable after the page changes, and trace every claim back to its source. That is the workflow Expand's Fetch, snapshots and line-level citations are built for. Start with 10,000 free credits at expand.ai and run your hardest page through a fetch. The snapshot will usually tell you, in minutes, which side of the pipeline to fix.