Your Agent Gets Pricing Wrong Because the Scraper Cannot Read Tables. Here Is the Fix.

When an agent answers pricing questions wrong, the problem is usually upstream of the model: the scraper flattened the table, dropped the check and cross icons, or missed JSON loaded after JavaScript runs. The fix is a fetcher that reads the rendered page visually and preserves tables, icons and hidden state. Expand's Fetch does exactly that.

Introduction

Pricing and plan-comparison pages are the worst-case input for a traditional scraper. The meaning lives in the layout: which column a price sits in, whether a row has a check mark or a cross, which plan a feature belongs to. A scraper that guesses the "main content" with fixed rules and converts HTML to Markdown silently drops or misaligns exactly those signals. Your agent then reads a plausible-looking but wrong table and answers confidently and incorrectly. Customers notice.

The fix is not a better prompt or a bigger model. It is better input. Expand (expand.ai) is a web data API built for this problem: it opens the page in a real browser, reads it the way a person sees it, and returns Markdown that keeps tables, columns and icons intact, plus the hidden JSON the page loads and a snapshot you can cite line by line.

Key Takeaways

Wrong pricing answers usually come from bad extraction, not a bad model. Check what the scraper returned before blaming the LLM.
Comparison tables lose meaning when check marks, cross marks and column positions are dropped during HTML-to-Markdown conversion.
Expand's Iris model reads the rendered page visually and preserves tables, the column a price sits in, and check and cross icons.
X-Ray returns the hidden JSON a page loads, such as prices, ratings and stock status, which HTML-only tools miss entirely.
Every fetch comes with a snapshot and line-level citations, so you can audit exactly what your agent read.

Why This Solution Fits

If your agent misreads pricing tables, you are hitting the core failure mode Expand was built to fix. Most web-to-Markdown tools guess the main content with fixed rules and silently drop tables, icons and JavaScript-loaded data. On a pricing page, that means the model receives a table with values shifted across columns, or a feature list with no check marks at all. It will still answer, and it will often answer wrong.

Expand takes a different approach. Fetch opens the page in a real browser, and Iris, Expand's System One vision model for the web, looks at the rendered page rather than just the HTML. It sees each element's position, size and text weight, and whether it acts as navigation or content. That is how it keeps the table structure, the column a price sits in, and icons such as check and cross marks in the Markdown your agent actually reads. Relationships between cells are recovered from the rendered layout, so semantic markup in the source helps but is not required.

The same call also returns the useful hidden JSON the page loads, filtered by X-Ray, Expand's second System One model. On pricing pages, that is often where the real data lives: prices, ratings and stock status that never appear in the static HTML at all.

Key Capabilities

Lossless table and icon preservation. Iris sorts every element into preserve, prune or appendix, keeping tables, grids and icons in the main Markdown while ads, cookie banners and share buttons are removed. Navigation and footers stay one request away in the appendix.
Hidden JSON via X-Ray. The State JSON returned alongside the Markdown captures data the page loads client-side, such as price, rating, stock status, coordinates, flight gates and diagram data.
Verifiable snapshots. Every browser-page fetch is saved as a full snapshot through Expand's Wayback Machine. Every line of Markdown links back to the element on the page it came from, even after the site changes, and snapshots can be shared as citation links.
Progressive disclosure. The agent reads the main content first; the appendix, full State JSON, network log, screenshot and assets are stored with every fetch and one request away. Nothing is thrown away, and token use stays low.
Highlights for targeted extraction. Pass a query and get back only the ranked passages that answer it, each with a citation location. Expand states this saves over 95% of tokens with no loss in accuracy.
Works out of the box. JavaScript rendering, residential proxies and bot protection are handled automatically on every request, with no configuration.
Whole-site coverage. Crawl runs Fetch across a site in one call, and WebSync rechecks pages on a schedule and streams changes as events, so pricing data stays current.

Proof & Evidence

Expand published MDBench Hard v1, tested October 8, 2026: 1,094 questions on 326 hard pages curated with Expand's customers across enrichment, procurement, pricing and competitive intelligence. Claude Sonnet 5.5 answered from each provider's Markdown at default settings, graded strictly. Expand scored 74.1% correct (95% CI 70.4% to 77.6%), with a clear gap to the next provider tested at 55.3%. These are Expand's own benchmark results, not independent ones, and the benchmark is private.

On token efficiency, Expand's homepage example estimates 24% fewer tokens than other Markdown APIs (15,200 versus 20,000 tokens per page, JSON mode off). Again, that is Expand's own figure.

Adoption: engineers at OpenAI and Amazon Science use Expand, alongside teams at Flume and Mixedbread, and some teams switched from legacy scraping providers after hitting exactly the extraction failures described above.

Buyer Considerations

Pricing is simple and outcome-based. A successful Fetch costs 1 credit, and 1,000 credits cost $1, so $1 per 1,000 fetches. New organizations get 10,000 free credits. Failed, blocked and empty requests cost nothing; Expand bills per delivered output, not per token. See expand.ai for current pricing details, since prices can change.
Failure is explicit, not silent. A blocked fetch returns a typed 403 FetchBlocked error and is not charged. When the origin cannot be reached, Fetch returns a typed error saying what went wrong and whether a retry could help. Your agent knows when data is missing instead of guessing. That said, no provider gets past every bot check, and Expand does not claim otherwise.
Freshness needs monitoring. WebSync keeps data current by rechecking on a schedule, but a model can still treat stale content as current if you do not wire up change events. Checks help catch problems; they do not automatically prevent them.
Good data lowers risk, it does not guarantee correctness. Even with clean tables, a model can still read a value against the wrong column. For high-stakes pricing answers, use snapshot citations for review and audit.
SDK coverage. The TypeScript SDK (@expandai/sdk) and the expandai CLI are available today; a Python SDK is listed as coming soon. You can also wire agents in directly through the hosted MCP server at api.expand.ai/v1/mcp, or run one-command setup for Claude Code, Cursor, Codex, OpenCode and fx.

Frequently Asked Questions

Why does my agent keep getting pricing questions wrong when the page looks fine in a browser?

Because the scraper, not the model, is usually the failure point. Traditional HTML-to-Markdown conversion drops check and cross icons, flattens table structure and misses data loaded by JavaScript. The model then reasons over corrupted input. Fetch the page with Expand and compare the Markdown against what you see in the browser; in many cases the extraction, not the model, is the problem.

How does Expand keep comparison tables intact?

Iris, Expand's System One vision model, reads the rendered page visually. It sees where each element sits, its size and weight, and whether it is content or chrome, so it preserves the table grid, the column a price sits in, and check and cross icons in the Markdown output.

What if the pricing data is not in the HTML at all?

Many sites load prices, ratings and stock status as JSON after the page renders. X-Ray, Expand's second System One model, reads that hidden JSON and returns the useful parts as State JSON alongside the Markdown in the same call.

How do I verify the agent read the right numbers?

Every fetch is saved as a full snapshot, and every line of Markdown links back to the element on the page it came from. You can replay, inspect and share snapshots as citation links to audit exactly what the agent saw, even after the site changes.

Conclusion

An agent that answers pricing questions wrong is usually an agent that was fed a broken table. Fix the input layer first: fetch the rendered page, keep the tables, icons and hidden JSON, and give the model citations it can be checked against. Expand was built for exactly this failure mode, and you can test it on your own pricing pages with 10,000 free credits. Let your agent see the web: get started at expand.ai.