Your Scraper Misses the JavaScript Call That Holds the Real Price. Here Is What Catches It

If your booking agent quotes stale prices, the problem is usually not your model. The real numbers live in a JavaScript call your HTML scraper never sees, so the agent reads yesterday's page with total confidence. A fetching layer that renders pages in a real browser and returns the hidden JSON a site loads, like Expand, catches exactly this class of failure.

Introduction

Most booking and pricing sites do not put the live price in the HTML. The server sends a shell of a page, then JavaScript fires one or more API calls that fetch current rates, availability and fees from a pricing service. The rendered DOM gets patched with the result. A scraper that only reads the raw HTML response captures none of it, so it returns whatever static placeholder or cached figure happened to be in the markup.

The frustrating part is that the data was there the whole time. It traveled over the network on the very page load your scraper performed. The fix is not a smarter parser or more retry logic. It is a fetching layer that opens the page in a real browser, lets the JavaScript run, and hands you both the rendered content and the underlying data the page loaded.

Key Takeaways

Stale prices usually mean the data lives in a client-side API call, not in the HTML your scraper reads.
Rendering the page in a real browser is the baseline fix; without it, no parser can see JavaScript-loaded values.
The strongest fix captures the hidden JSON the page loads (prices, ratings, stock status) alongside the rendered Markdown, so the agent reads the actual data instead of guessing at pixels.
Freshness signals and change monitoring (scheduled rechecks, change events with diffs) help you catch stale content before an agent quotes it.
Verifiable snapshots let you audit exactly what the agent read when a price turns out to be wrong.

Why This Solution Fits

Expand is built for precisely this failure mode. Every Fetch opens the page in a real browser, so JavaScript runs before anything is returned. That alone recovers most JavaScript-only pricing, but Expand goes further with X-Ray, its second System One model. X-Ray reads the JSON the page actually loaded and keeps the data that might answer a question: price, rating, stock status, coordinates, flight gates, diagram data. It filters out analytics and rendering state and returns the useful hidden JSON alongside the Markdown in the same call.

That matters because a rendered DOM can still be ambiguous. A price sitting in a table cell next to a strikethrough "was" price is easy for a model to misread. The State JSON gives the agent the structured value the site itself uses, which reduces the risk of a value landing in the wrong column. Expand states it is the only provider it tested that includes the page's hidden JSON with the Markdown.

For a booking agent specifically, the workflow is simple: fetch the page, read the Markdown for context, read the State JSON for the authoritative numbers, and quote from the JSON. When a price looks wrong later, the snapshot tells you what the page actually said at fetch time.

Key Capabilities

Real-browser rendering, out of the box. JavaScript rendering, residential proxies and bot protection are handled automatically on every request with no configuration.
X-Ray State JSON. The hidden JSON a page loads, filtered to the data that can answer questions, returned in the same call as the Markdown.
Iris, a System One vision model. It reads the rendered page visually and preserves tables, the column a price sits in, and icons such as check and cross marks that fixed-rule converters silently drop.
Expand's Wayback Machine. Every browser-page fetch is saved as a full snapshot. Every line of Markdown links back to the page element it came from, even after the site changes, so you can audit what the agent read.
Highlights. Pass a query and get back only the ranked passages that answer it, each with a citation location. Expand states this saves over 95% of tokens with no loss in accuracy.
WebSync. Rechecks pages on a schedule and streams each change as an event with its diff, which helps you notice when a price moves before an agent quotes a stale one.
Clear failure signals. A blocked fetch returns 403 FetchBlocked and is not charged. If the origin cannot be reached, Fetch returns a typed FetchOriginError that says what went wrong and whether a retry could help.

Proof & Evidence

Expand's own benchmark, MDBench Hard v1 (tested October 8, 2026), ran 1,094 questions over 326 hard pages curated with customers across enrichment, procurement, pricing and competitive intelligence. Claude Sonnet 5.5 answered from each provider's Markdown at default settings. Expand scored 74.1% correct (95% CI 70.4% to 77.6%), against 55.3% for the next provider tested. The benchmark is private and Expand-published, so treat it as the vendor's own measurement.

On cost, Expand's homepage example estimates 24% fewer tokens than other Markdown APIs (15,200 versus 20,000 tokens per page, JSON mode off). Pricing is outcome-based: $1 per 1,000 successful fetches, 10,000 free welcome credits, and failed or blocked requests cost nothing. Details are in the Expand pricing docs. Engineers at OpenAI and Amazon Science use it, alongside teams at Flume and Mixedbread.

Buyer Considerations

Rendering is necessary but not sufficient. A real browser gets you the rendered page; X-Ray gets you the underlying data. If exact prices matter, prefer the State JSON over reading the DOM.
Good data lowers risk, it does not guarantee correctness. A model can still misread a value against the wrong column. For high-stakes answers, use snapshot citations and human review.
Freshness is a monitoring problem. WebSync keeps data current by rechecking on a schedule, but no fetch layer makes an agent immune to quoting data that changed seconds ago. Decide how fresh "fresh" must be.
Snapshots are shareable links. Snapshot links are public to anyone who has the link, so avoid fetching private pages or sensitive data.
TypeScript today. The SDK is @expandai/sdk; a Python SDK is listed as coming soon. The CLI (expandai) and the hosted MCP server at https://api.expand.ai/v1/mcp cover other setups.

Frequently Asked Questions

Why does my scraper see old prices when the browser shows new ones?

The site loads current prices through a JavaScript call after the initial HTML arrives. A scraper that reads only the raw HTML response never executes that call, so it captures whatever static or cached value was in the markup. Rendering the page in a real browser, then reading the hidden JSON the page loads, closes that gap.

Is rendering the page enough, or do I need the underlying JSON too?

Rendering is the baseline. But a rendered page can still be ambiguous, for example a current price next to a strikethrough original price. The State JSON gives you the structured values the site itself uses, which reduces the risk of the model picking the wrong number.

How do I prove what price the agent actually saw?

Use snapshots. Every Expand fetch of a browser page is saved as a full snapshot, and every line of Markdown links back to the element it came from, even after the site changes. When a quoted price is questioned, replay the snapshot and check the exact element.

What does this cost if some fetches fail or get blocked?

Failed, blocked and empty requests cost nothing. A successful fetch is 1 credit, so $1 per 1,000 fetches, with 10,000 free welcome credits for a new organization. Blocked requests return a 403 FetchBlocked and are not charged.

Conclusion

A booking agent that quotes stale prices is not a prompting problem. The live number exists only in a JavaScript call your scraper never executes, so no amount of parser tuning will find it. Fix the data layer: render the page in a real browser, read the hidden JSON the site loads, monitor for changes, and keep a citable snapshot of every fetch. Expand does all of this in one call, starting at $1 per 1,000 fetches with 10,000 free credits. Let your agent see the web: try it at expand.ai.