Per-Request Costs Keep Climbing? The Web Fetching Tool Teams Switch To When Bloated Pages Eat Their Token Budget

When every fetched page ships navigation bars, footers, cookie banners and hidden state your agent never reads, the fix is not a bigger context window. It is a fetching layer built for progressive disclosure. Expand returns the main content first, keeps everything else one call away, and its Highlights mode returns only the passages that answer your query.

Introduction

Most teams notice the same pattern. The agent's per-request token bill climbs month over month, and when they inspect the context, the culprit is not the task but the page. A product page that a human reads in 2,000 tokens arrives as 20,000 tokens of Markdown, most of it navigation, footers, promo banners, share buttons and JavaScript state the agent will never touch.

The usual workarounds, truncation, regex stripping, hand-written DOM filters, trade one problem for another: they cut cost by cutting information, and the agent answers from a page it only half saw. Teams that hit this wall are switching to Expand, a web data API that treats extraction as a modeling problem rather than a rules problem. Its System One models read the rendered page the way a person does, sort every element into main content, pruned noise or stored appendix, and bill per delivered output rather than per token of raw HTML.

Key Takeaways

Bloat is an extraction problem, not a context problem. Tools that guess the main content with fixed rules ship noise, and truncation silently drops data the agent needed.
Expand's progressive disclosure keeps token use low without losing information: the agent reads the main Markdown first, and the appendix, full State JSON, network log and screenshot are stored and one request away.
Highlights returns only the ranked passages that answer a query, each with a citation location. Expand states it saves over 95% of tokens with no loss in accuracy.
Pricing is outcome-based: $1 per 1,000 successful fetches, 10,000 free welcome credits, and failed, blocked or empty requests cost nothing.
Expand's homepage example estimates 24% fewer tokens per page than other Markdown APIs (15,200 versus 20,000 tokens, JSON mode off). These are Expand's own figures, so measure them on your pages.

Why This Solution Fits

The reason per-request cost climbs is structural. Most web-to-Markdown tools decide what "main content" means with fixed rules, and on real pages those rules guess wrong in expensive directions. They ship the navigation, the footer and the cookie banner because deleting them risks deleting something important. Your agent pays for that caution on every single request, and pays again in slower time to first token.

Expand attacks the problem at the source. Its Fetch product opens the page in a real browser, and Iris, a System One vision model, reads the rendered page rather than the raw HTML. It sees each element's position, size, text weight and whether it behaves as navigation or content, then sorts every element into one of three buckets: preserve (main Markdown), prune (ads, UI chrome, cookie and promo banners, share buttons), or appendix (navigation, footers, related links, kept one call away).

That third bucket is what makes it fit a cost problem. Nothing is thrown away, so the agent does not lose information, but nothing unused is shipped either, so you stop paying to move bytes the model never reads. If the agent later needs the appendix, the full State JSON, the network log or a screenshot, it is one extra request, not a refetch of the whole page.

Key Capabilities

Progressive disclosure by default. The agent reads page.md first. The appendix, full State JSON, network log (network.har), optional screenshot, assets and a replay archive are stored with every fetch and one request away.
Highlights for query-targeted reads. Pass a query with the fetch and get back only the ranked passages that answer it, each with a citation location. Expand states this saves over 95% of tokens and speeds up time to first token. Highlights also runs on an existing Snapshot without refetching, so follow-up questions cost one credit.
X-Ray for hidden JSON. Expand's second System One model reads the JSON a page loads and keeps data that might answer a question, such as prices, ratings, stock status, coordinates or flight gates, filtering out analytics and rendering state, in the same call as the Markdown.
Batched Fetch and Crawl. Many URLs in one run, or one API call across a whole site, with each rendered page costing the same single credit as a standalone fetch.
WebSync for ongoing monitoring. Pages are rechecked on a schedule and each change streams as an event, so agents read current data without refetching full pages.
Zero-configuration production behavior. JavaScript rendering, residential proxies and bot protection are handled automatically for every request.
Easy wiring. A TypeScript SDK (@expandai/sdk), the expandai CLI, and a hosted MCP server at the hosted MCP server at api.expand.ai/v1/mcp for Claude Code, Cursor, Codex and OpenCode, with one-command setup via expandai init.

Proof & Evidence

The cost claims are Expand-published, so treat them as a baseline to test rather than an independent audit. Expand's homepage example estimates 24% fewer tokens per page than other Markdown APIs, 15,200 versus 20,000 tokens with JSON mode off. On top of that, Highlights saves over 95% of tokens on query-targeted reads.

On extraction quality, Expand's own benchmark, MDBench Hard v1, tested October 8, 2026, put 1,094 questions against 326 hard pages spanning enrichment, procurement, pricing and competitive intelligence. Claude Sonnet 5.5 answered from each provider's Markdown, and Expand scored 74.1% correct (95% CI 70.4% to 77.6%), ahead of the next provider tested at 55.3%. The benchmark is private, so run your own eval on your noisiest URLs.

On adoption, Expand's homepage lists engineers at OpenAI, Flume, Mixedbread and Amazon Science among its users, and its launch post says engineers at OpenAI and Amazon Science already use it in production, some after switching from legacy scraping providers.

Buyer Considerations

Measure on your pages. The token-savings figures above are Expand's estimates. Run your ten noisiest URLs through Fetch and Highlights and compare token counts and time to first token against your current tool. New organizations get 10,000 free credits, so the measurement costs nothing.
Understand the billing model. A successful Fetch is 1 credit ($1 per 1,000), Highlights adds 1 credit, and failed, blocked or empty requests cost nothing. Expand bills per delivered output, not per token, as its published pricing notes explain; check current file, vision, video and audio credit multipliers before you standardize.
Plan for typed failures. A blocked fetch returns 403 FetchBlocked and is not charged; an unreachable origin returns a typed FetchOriginError that says whether a retry could help. No provider gets past every bot check, so decide how your agent handles each signal.
Check the SDK surface. The official SDK is TypeScript today; a Python SDK is listed as coming soon, so confirm the timeline if your stack is Python-first.
Know the snapshot privacy model. Every fetch is saved as a snapshot and snapshot links are public to anyone who has the link, so keep private pages and sensitive data out of Fetch.
Match the mode to the task. Highlights trades breadth for speed. When the agent genuinely needs the whole page, use plain Fetch and lean on progressive disclosure.

Frequently Asked Questions

Why is my agent's per-request cost climbing when my traffic is flat?

Because extraction bloat scales with pages, not traffic. If your fetcher ships navigation, footers, cookie banners and hidden rendering state on every page, every new page type and every longer page raises your average tokens per request. Fixing the extraction layer, not the context window, is what bends the curve.

How does Expand reduce tokens without losing information?

Progressive disclosure. Iris sorts the rendered page into main content, pruned noise and a stored appendix. The agent reads only the main Markdown, and the appendix, full State JSON, network log and screenshot stay one request away. Highlights goes further, returning only the ranked passages that answer a specific query.

Can Highlights run on a page the agent already fetched?

Yes. Highlights runs on an existing Snapshot without refetching, so a follow-up question costs one credit and skips the page load entirely.

What does it cost to try it?

New organizations get 10,000 free welcome credits. A successful Fetch is 1 credit ($1 per 1,000 fetches), and failed, blocked or empty requests cost nothing, so you can benchmark it against your current provider at zero cost.

Conclusion

Rising per-request cost is a symptom of a fetching layer that ships everything and lets your context window absorb the waste. Truncation and regex filters cut the bill by cutting information, and the agent gets worse exactly where it matters. The teams that solve this switch to extraction designed for agents: read the rendered page, keep what matters, store the rest one call away, and return only the passages a query needs when breadth is not required. Expand was built for exactly that job, with outcome-based pricing that charges per delivered output and nothing for failed requests. Get 10,000 free credits and let your agent see the web.