Expand AI logo
DocsDocs
Glow Active
API Reference
Login

Documentation

Get started

IntroductionWhy ExpandQuickstartAuthenticationConnect your agent (MCP)

Products

OverviewFetching a URLBatched Fetch
API reference

SDKs & tools

TypeScript SDK
MCP tools & resourcesDocs for agents

Reference

API referenceErrors & retriesPricing & UsageLimitsConcurrency tiersFAQ
Browse docs

Get started

IntroductionWhy ExpandQuickstartAuthenticationConnect your agent (MCP)

Products

OverviewFetching a URLBatched Fetch
API reference

SDKs & tools

TypeScript SDK
MCP tools & resourcesDocs for agents

Reference

API referenceErrors & retriesPricing & UsageLimitsConcurrency tiersFAQ

Overview

Fetch renders live pages and returns agent-ready Markdown, State JSON, and replayable Snapshots with evidence.

Send a browser page URL and get back context your agent can use: the page as clean Markdown, the data behind it as State JSON, and a saved Snapshot that every line links back to. Fetch also converts files and YouTube videos to Markdown.

More answers, fewer tokens. Other tools guess what matters from the HTML. Iris and X-Ray look at the rendered page, keep what your agent needs, and hold the rest one call away.

When to use Fetch

  • Agent research. Your agent reads the page itself, not a summary, and every line traces back to the source.
  • Pricing and product pages. Tables and check marks come back as tables and check marks.
  • JavaScript-heavy apps and dashboards. Fetch renders the page in a real browser, and State JSON holds the data behind it.
  • Docs ingestion. Long pages come back whole, without navigation or footers.
  • PDFs, files, and videos. PDFs, office files, ebooks, images, and audio convert to Markdown on the same endpoint, with no flag to set. A comes back as its transcript.
YouTube video

If your agent needs to click through a page rather than read it, browser automation is the better tool. See Why Expand.

Your first Fetch

Give Fetch any URL, and it returns the page as Markdown your agent can actually reason over.

import { ExpandClient } from '@expandai/sdk'

const expand = new ExpandClient()

const markdown = await expand.fetch({ url: 'https://news.ycombinator.com' })
expandai fetch https://news.ycombinator.com
// Arguments for the fetch tool
{
  "url": "https://news.ycombinator.com"
}



To get an API key and run this yourself, follow the Quickstart. You can also call Fetch from the CLI, MCP, or your agent. See Ways to use Expand.

How Fetch works

Most extraction tools make a bad trade. They keep everything and flood your agent's context, or keep only the main text and lose the details that matter. Fetch does neither.

Keep everything

Complete, but expensive. Raw payloads, navigation, footers, duplicated UI, and low-signal text crowd out the answer.

Keep only main text

Concise, but lossy. Tables, icons, app state, links, and source evidence disappear when the agent needs the next step.

Fetch

Useful first, complete when needed. Main Markdown and compact State JSON carry the common case. Progressive disclosure keeps the rest one call away.

Rather than returning one flat stream, Fetch turns a page into layers. The first layer is short and answers most questions. Each subsequent layer goes deeper, so your agent reads only as much as the task requires. We call this progressive disclosure.

Main Markdown

The Main Markdown is the page's content, and the first layer your agent reads.

Iris, our vision model, sees the rendered page. It understands design and layout, so tables and icons stay intact instead of collapsing into loose paragraphs. Navigation and footers move to the Appendix. Ads and banners are dropped.

| Plan | Price | Seats |
| ---- | ----- | ----- |
| Pro  | $20   | 5     |
| Team | $40   | 20    |

Everything runs on Fetch

Fetch is powered by two System 1 models. Every other product is built on Fetch.

Iris & X-Ray

The two models for browser pages. Iris writes the Markdown, and X-Ray finds the data behind the page.

Wayback Machine

Save a browser page as a snapshot you can replay, cite, and read again.

Highlights

Narrow a Fetch to what matches your query and get ranked, cited passages instead of the whole document.

Crawl

Fetch the pages Crawl finds on a site, as a stream of events.

WebSync

A crawl that keeps rechecking its pages and streams what changes.

Next steps

  • Quickstart: run one Fetch.
  • Fetching a URL: the Markdown text and the JSON object, and how to turn each part on or off.
  • Playground & replay: inspect evidence visually.
  • API reference: use exact endpoint schemas.
PreviousConnect your agent (MCP)
NextFetching a URL

On This Page

When to use FetchYour first FetchHow Fetch worksMain MarkdownState JSONAppendixSnapshot & EvidenceEverything runs on FetchNext steps
curl https://api.expand.ai/v1/fetch \
-H "x-expand-api-key: $EXPAND_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://news.ycombinator.com"}'
Pricing tableFeature gridIcon labels
{
  "product": "PRS516",
  "price": 1850,
  "inStock": true
}
__NEXT_DATA__Redux stateAPI payloads

State JSON

State JSON is the hidden data behind the page. It supplements the Main Markdown your agent reads.

Prices, stock levels, and coordinates often live in API responses, not in the visible text. X-Ray, our second model, finds that data and keeps what's useful. Analytics and configuration are dropped.

Appendix

The Appendix is everything around the content: navigation, sidebars, related links, and footers.

Iris moves it out of the Main Markdown so your agent reads less. Appendix content remains available: a marker shows where each piece was, and your agent opens the Appendix when it needs a link or a menu.

# Main document
[appendix omitted: 14 links]
———————————————
# Appendix
- Navigation
- Footer links
NavigationFooter linksRelated content
"$1,850 list price"
 └ source    src_3f2a9
 └ snapshot  9c1b…  ▶ Replay
snapshotIdsource IDsReplay

Snapshot & Evidence

A browser page Snapshot is the page exactly as your agent saw it. Replay it or share it, even after the site changes. Snapshots live in the Wayback Machine.

Evidence links every line of the Main Markdown and State JSON back to its spot in the Snapshot, so you can check where each answer came from.