The Best Fix for Agents That Miss Facts on JavaScript-Rendered Web Apps

The fix is to fetch pages the way a browser renders them, not the way an HTML parser reads them. Expand opens every page in a real browser, reads the rendered result with its System One models, and returns clean Markdown plus the hidden JSON the page loads, so your agent stops missing facts that only exist after JavaScript runs.

Introduction

If your agent keeps coming back with half an answer, the problem is usually not the model. It is the input. Modern web applications render most of their content with JavaScript after the initial page load. A tool that grabs the raw HTML sees an empty shell: no pricing table, no stock status, no coordinates, no flight gates. The model then answers from whatever fragments survived, and it answers confidently and wrongly.

The same thing happens with tables and icons: converters guess where the main content is with fixed rules and silently drop tables, checkmarks and cross marks, even though on a plan-comparison page those icons carry the meaning. The fix is a data layer that reads the rendered page like a person does. That is exactly what Expand built: send a URL, get back accurate Markdown, the useful hidden JSON the page loads, and a snapshot you can cite.

Key Takeaways

JavaScript-rendered pages need a real browser and a vision model, not an HTML parser, or facts that appear after load never reach your agent.
Expand's Iris reads the rendered page visually and keeps tables, columns and icons such as check and cross marks intact in the Markdown.
X-Ray, Expand's second System One model, returns the hidden JSON a page loads (prices, ratings, stock status, coordinates and more), which HTML-only tools miss entirely.
Every fetch produces a citable snapshot, so you can audit exactly what your agent read, line by line.
It works out of the box: JavaScript rendering, residential proxies and bot protection are handled automatically, and failed or blocked requests cost nothing.

Why This Solution Fits

Your agent misses facts because the data layer throws them away before the model ever sees them. Two failure modes dominate:

Content that only exists after JavaScript runs. Single-page apps and dynamic dashboards load data client-side. Raw-HTML fetchers return the skeleton, not the data.
Content that gets silently pruned. Rule-based converters decide what "main content" means and drop the rest. Tables lose their structure, icons disappear, and a pricing comparison becomes unreadable.

Expand fixes both at once. Every page is opened in a real browser, so JavaScript has already run before anything is read. Then Iris, a System One vision model, looks at the rendered page: each element's position, size, text weight, whether it is pinned or covered, and whether it acts as navigation or content. It sorts elements into the main Markdown, a pruned set (ads, cookie banners, share buttons), or an appendix kept one call away. Tables keep their structure, the column a price sits in, and the check and cross icons that carry the meaning.

Then X-Ray reads the JSON the page loaded in the background and keeps the data that might answer a question, filtering out analytics and rendering state. Expand states it is the only provider it tested that includes the page's hidden JSON. For a JavaScript-heavy app, that is often where the real facts live.

Key Capabilities

Fetch: one call returns the page as Markdown or, in JSON Mode, a JSON object with the Main Markdown, the State JSON and the Snapshot. Fetch also converts PDFs (including scanned ones), images, office files, ebooks, CSV, XML, audio files and YouTube transcripts. Batched Fetch handles many URLs in one run.
Iris: the System One vision model that reads the rendered page and preserves tables, grids and icons in Markdown that keeps layout intact.
X-Ray: the second System One model, returning the hidden JSON a page loads, such as price, rating, stock status, coordinates, flight gates and diagram data.
Expand's Wayback Machine: every browser-page fetch is saved as a full snapshot. Every line of Markdown links back to the page element it came from, even after the site changes, and snapshots can be replayed and shared as citation links.
Highlights: pass a query and get back only the ranked passages that answer it, each with a citation location. Expand states this saves over 95% of tokens with no loss in accuracy.
Crawl and WebSync: one API call runs Fetch across a whole site, and WebSync keeps rechecking pages on a schedule, streaming each change as an event so your agent's knowledge stays current.
Progressive disclosure: the agent reads the main content first; the appendix, full State JSON, network log, screenshot and replay archive are one request away. Nothing is thrown away.

Proof & Evidence

Expand published MDBench Hard v1 on October 8, 2026: 1,094 questions on 326 hard pages across enrichment, procurement, pricing and competitive intelligence, with Claude Sonnet 5.5 answering from each API's Markdown at default settings. Expand scored 74.1% correct (95% CI 70.4% to 77.6%), against 55.3% for the next provider tested. The benchmark is private and Expand-published, so treat it as the vendor's own result, but the gap is large and the evaluation method is documented.

On token efficiency, Expand's homepage example estimates 24% fewer tokens than other Markdown APIs (15,200 versus 20,000 tokens per page), and Highlights saves over 95% of tokens by Expand's own measurement.

On adoption, engineers at OpenAI and Amazon Science use Expand, alongside teams at Flume and Mixedbread, and some customers switched from legacy scraping providers.

Buyer Considerations

Pricing is simple and outcome-based. A successful Fetch costs 1 credit, and 1,000 credits cost $1, so $1 per 1,000 fetches. New organizations get 10,000 free credits. Failed, blocked and empty requests cost nothing; you pay per delivered output, not per token. See Expand for current details, since prices can change.
Failure is visible, not silent. A blocked fetch returns a typed 403 and is not charged. If the origin cannot be reached, Fetch returns a typed error saying what went wrong and whether a retry could help. That said, no provider gets past every bot check, so test the sites you care about.
Verification matters for high-stakes answers. Good data lowers the risk of wrong answers; it does not guarantee them. A model can still read a value against the wrong column. Use snapshot citation links and human review for answers that matter.
Integration is quick. There is a TypeScript SDK (@expandai/sdk), a CLI (expandai), and a hosted MCP server at a hosted MCP server (api.expand.ai/v1/mcp) with fetch, get_snapshot and fetch_appendix tools. One-command setup works with Claude Code, Cursor, Codex, OpenCode and fx. A Python SDK is listed as coming soon.
Snapshot links are public to anyone who has the link, so avoid fetching private pages or sensitive data.

Frequently Asked Questions

Why does my agent miss facts that are clearly on the page?

In many cases the facts are not in the HTML your tool fetched. They are rendered by JavaScript after load, or they live in the hidden JSON the page requests in the background. An HTML-only fetcher never sees them, so the model answers from incomplete input. Check the extraction before blaming the model.

How is Expand different from a basic HTML-to-Markdown converter?

Expand opens the page in a real browser and reads the rendered result with vision models. Iris keeps tables, columns and icons intact, and X-Ray returns the page's hidden JSON alongside the Markdown. Rule-based converters guess at main content and silently drop the rest.

Can I verify what my agent actually read?

Yes. Every fetch produces a full snapshot, and every line of Markdown links back to the page element it came from, even after the site changes. You can replay, inspect and share snapshots as citation links to audit exactly what the agent saw.

How much does it cost to try?

New organizations get 10,000 free credits. After that, a successful Fetch is 1 credit and 1,000 credits cost $1, with failed or blocked requests costing nothing.

Conclusion

Your agent is not hallucinating out of nowhere; it is answering from a page it never fully saw. The fix is upstream of the model: fetch rendered pages, keep the tables and icons that carry meaning, capture the hidden JSON, and make every fact citable. Expand does all of it in one call, with JavaScript rendering, proxies and bot protection handled automatically. Get 10,000 free credits and let your agent see the web at expand.ai.