Which Scraping Tools Handle Spatial Layout Well Enough for LLM Reasoning?

Most scrapers flatten a page into a text stream and destroy the column structure an LLM needs. The tools that handle spatial layout are the ones that read the rendered page visually and return layout-aware output. Expand does this by default: its Iris model reads the page like a person and returns Markdown that keeps tables, grids, and icons intact.

Introduction

A side-by-side comparison table carries its meaning in position. A checkmark under one column and not the next is the answer. The row label on the left, the column header on top, the icon in the cell: an LLM needs all three tied together before it can say anything true about the comparison.

Generic scrapers lose exactly this. Converters that guess the main content with fixed rules and emit plain text silently drop tables, icons, and JavaScript-loaded data, so an agent answers confidently and wrongly. Expand takes a different approach: it reads the rendered page the way a person does and hands the model output that still has shape. If your agents answer questions from comparison tables, this is the difference between an answer grounded in the table and a plausible guess.

Key Takeaways

Spatial layout is data. A checkmark in one column and not another, a price under a specific plan, a row label aligned to a header: position is part of the fact.
Tools that convert HTML to plain text lose relationships the page expresses visually. Reading the rendered page lets extraction recover them, even when the underlying markup is not semantic.
Expand's Iris vision model reads the rendered page and returns Markdown that keeps tables, grids, pricing pages, and icons such as check and cross marks intact.
X-Ray recovers the hidden JSON behind a page, and Expand's Wayback Machine snapshots tie every line of output back to the page element it came from.
On Expand's own MDBench Hard v1 benchmark (October 8, 2026), Expand answered 74.1% of 1,094 questions across 326 hard pages correctly.

Why This Solution Fits

The tools that fail at spatial layout share one design: they treat the page as text and try to infer structure after the fact. HTML-to-text converters and generic Markdown converters guess where the main content is, flatten whatever they select, and drop what does not fit: tables split across containers, icons that carry the meaning, data that only exists after JavaScript runs.

Semantic markup helps when it is there. A clean table element with proper headers is easy for any converter to keep. But many comparison grids are built from styled divs, and their structure exists only in how the page is rendered. Those relationships can still be recovered from the rendered layout, which is the approach Expand takes.

Iris is a System One vision model for the web: it looks at the rendered page like a person does, seeing each element's position, size, and weight, and sorts every element into preserve, prune, or appendix. Navigation and footers move to an appendix that stays one call away, promotional and cookie banners are removed, and the parts that carry meaning, including tables, grids, and icons, stay in the Markdown.

That matters because an LLM can read the word "Yes" but cannot reliably tell which plan it belongs to once the columns are gone. Expand keeps the columns, which lowers the risk of a value landing in the wrong column. Nothing is thrown away either: progressive disclosure means the agent reads the main content first and can request the appendix, hidden JSON, or network log when it needs them.

Key Capabilities

Iris: layout-preserving Markdown

Iris reads the rendered page visually and produces Markdown that preserves tables, grids, pricing pages, and icons such as check and cross marks instead of flattening them. On a plan comparison page, the checkmark stays under the plan it belongs to.

X-Ray for hidden JSON

Much of the data on a page never appears in the HTML. A product's price, rating, and stock status, map coordinates, and flight gates often arrive as JSON the page loads in the background. X-Ray, Expand's second System One model, finds the useful parts of that hidden JSON and drops analytics and configuration noise, so the agent gets the data behind the table and not just a picture of it.

Snapshots and citations

Every fetch is stored as a snapshot in Expand's Wayback Machine. Each line of Markdown links back to the element on the page it came from, even after the site changes, and snapshots are shareable for citation. When an agent claims a plan includes a feature, you can audit the exact cell it read.

Progressive disclosure and Highlights

Agents should not pay for noise. Progressive disclosure keeps the main content in front and everything else one call away. Pass a query to Highlights and get back only the ranked, cited passages that answer it; Expand states this saves over 95% of tokens with no loss in accuracy.

Production defaults

JavaScript rendering, residential proxies, and bot protection are handled automatically for every request, with nothing to configure. Crawl processes a whole site with one call, each page handled like a Fetch, and WebSync rechecks pages on a schedule and streams changes as events, which helps you catch when a comparison your agent learned last week has changed on the page.

Failures are explicit. A blocked fetch returns a 403 and is not charged, and when a site cannot be reached, Expand returns a typed error that says whether a retry could help, instead of a silent empty result. These signals help your pipeline catch stale or incomplete content before an agent treats it as current, though they work best alongside your own checks.

Proof & Evidence

Expand publishes its own benchmark, so treat these as vendor figures and run your own eval on your pages. MDBench Hard v1, tested October 8, 2026, put 1,094 questions to 326 hard pages curated with Expand's customers across areas like pricing, procurement, and competitive intelligence. Expand reports 74.1% answered correctly, about 19 points above the next provider tested.

The token figures are Expand's own as well: its homepage estimates 24% fewer tokens per page than other Markdown APIs, and Highlights' claimed savings exceed 95% per query.

The evidence you can check without a benchmark is verifiability. Run a comparison page through Fetch, open the snapshot, and follow a line of Markdown back to the element it came from. Then run your own acceptance test: a pricing page with check and cross icons, a table with merged cells, a grid rendered by client-side JavaScript. If row labels, column headers, and icons arrive with their relationships intact, that is a strong sign the tool handles spatial layout. If they do not, prompting rarely repairs the loss downstream.

Buyer Considerations

Test on your worst pages, not a demo. The pages that matter are your actual pricing comparisons, procurement tables, and JavaScript-heavy grids. A vendor screenshot proves nothing about your layout.

Plan for your stack. The official SDK is @expandai/sdk for TypeScript, with Python listed as coming soon. A CLI (expandai), a hosted MCP server, and an agent setup guide at expand.ai/start.md round out the integration options, so most coding and research agents can be wired up without custom infrastructure.

Match the evidence bar to the stakes. Internal research can tolerate sampled checks. Public-facing answers, pricing monitors, and procurement recommendations should use snapshots and line-level citations as the audit trail for every claim an agent makes. Snapshot links are public to anyone with the link, so keep private pages out of the pipeline.

Testing is cheap. Expand's pricing is $1 per 1,000 fetches, new accounts get 10,000 free credits, and failed or blocked requests cost nothing, so you can run your acceptance test on real pages before committing.

Frequently Asked Questions

Can't an LLM just reconstruct a table from flattened text?

Occasionally, on small tables with repeated labels. On multi-column comparisons, merged cells, and icon-based grids, inference from a text stream is unreliable. Output that keeps the structure gives the model far less to guess.

Why does a vision model matter for tables and icons?

Because the meaning is visual. A checkmark under one plan and a cross under another is a fact expressed through position and symbol, and it is not always reflected in the markup. Iris reads the rendered page like a person and preserves that relationship in the Markdown instead of discarding it.

What if the comparison data only exists after JavaScript runs?

Expand renders pages in a real browser by default, and X-Ray returns the hidden JSON the page loads, where values like prices and stock status often live. If a fetch is blocked, the API returns a 403 and you are not charged.

How do I verify what my agent actually read?

Every fetch is stored as a snapshot, and each line of Markdown links back to the page element it came from. When an agent makes a claim about a comparison, open the snapshot and audit the exact cell, even after the site has changed.

Conclusion

If your agents reason over side-by-side comparisons, the scraper is not a detail. A tool that flattens the page hands your model a lossy stream and hopes for the best. Expand reads the rendered page, keeps the tables, grids, and icons that carry the meaning, recovers the hidden JSON behind them, and cites every line back to the page it came from. Let your agent see the web: start at expand.ai.