Expand AI logo
DocsDocs
Glow Active
API Reference
Login

Documentation

Get started

IntroductionWhy ExpandQuickstartAuthenticationConnect your agent (MCP)

Products

OverviewFetching a URLBatched Fetch
API reference
OverviewPlayground & replayProgressive disclosure

SDKs & tools

TypeScript SDK
MCP tools & resourcesDocs for agents

Reference

API referenceErrors & retriesPricing & UsageLimitsConcurrency tiersFAQ
Browse docs

Get started

IntroductionWhy ExpandQuickstartAuthenticationConnect your agent (MCP)

Products

OverviewFetching a URLBatched Fetch
API reference
OverviewPlayground & replayProgressive disclosure

SDKs & tools

TypeScript SDK
MCP tools & resourcesDocs for agents

Reference

API referenceErrors & retriesPricing & UsageLimitsConcurrency tiersFAQ

Progressive disclosure

Fetch once, then go back to the Snapshot for more. Read the original HTML, the network requests, or the whole archive without fetching the page again.

Fetch gives your agent the useful part first: the Main Markdown and State JSON. The Snapshot keeps everything else from the capture, and each part is one public GET away, with no API key. That is progressive disclosure: start small, and read more only when you need it.

The trail runs from the answer to the bytes. The Fetch response names the Snapshot's catalog, meta.resources. The catalog lists collections, such as the network requests, and artifacts, such as the original HTML. A collection holds records, an artifact offers a bounded preview, and both end in the saved content.

The catalog serves two different jobs:

  • Code. You fetched pages earlier and now want more from those Snapshots: the HAR to debug, the network records to find the API behind a page, the original HTML to re-parse.
  • Agents. The Markdown answered most of the question, but one value is missing. The agent looks it up in the saved HTML instead of fetching the page again.

Where the trail starts

Every Fetch response names the Snapshot's catalog. Markdown text carries a more URL in its frontmatter. The JSON object carries the same URL in meta.resources. Follow it exactly as given.

{
  "meta": {
    "resources": {
      "version": 1,
      "snapshotId": "01JPUBLIC01",
      "url": "https://api.expand.ai/v1/public/snapshots/01JPUBLIC01",
      "markdownUrl": "https://api.expand.ai/v1/public/snapshots/01JPUBLIC01?format=markdown"


Warning
Every URL on this page uses meta.resources.snapshotId, the public ID. It is not the meta.snapshotId that Highlights and the Playground use. See Two IDs.

Reading a Snapshot from code

Store meta.resources.snapshotId next to each Fetch result. Hours or days later, everything the capture saved is still readable by that ID, with no API key and no new browser session.

You needRead
The HTTP exchanges behind the page, to debug a bad capture or replay it elsewhereartifacts/har/content
The JSON API a page loads, so you can call it directly next timecollections.network for the request, or the HAR for every exchange
The original HTML, to re-parse with your own tools or diff against a later Fetchartifacts/html.original/content
The files the page referenced: images, PDFs, scripts

The catalog lists what was saved and where to read it:

import { ExpandClient } from '@expandai/sdk'

const expand = new ExpandClient()

const catalog = await expand.snapshots.get({ snapshotId: '<snapshot-link-id>' })
curl 'https://api.expand.ai/v1/public/snapshots/<snapshot-link-id>'
{
  "version": 1,
  "snapshotId": "01JPUBLIC01",
  "collections": {
    "evidence": "/v1/public/snapshots/01JPUBLIC01/evidence",
    "network": "/v1/public/snapshots/01JPUBLIC01/network",
    "assets": 











The links are paths on the API origin. The catalog has two kinds of entry:

  • Collections are lists, returned as { items, next? }. A network or asset record carries the captured request and response headers, status, and timing, plus an artifact link to the HAR and a content link to the saved body. A record's url says where the bytes came from. It is never a download target.
  • Artifacts are saved bytes. preview gives a bounded, base64-encoded look, and url gives the full content.
Note
Requests to the same URL share one content link, which returns the first match. For polling or repeated POSTs, read the HAR, which keeps every exchange. A preview answers 413 when the archive is over the server's budget. The content link still works, so fall through to the full download.

With the TypeScript SDK, one fetchJson runs the browser and everything after reads from the Snapshot:

import { ExpandClient, isFetchOriginError } from '@expandai/sdk'

const expand = new ExpandClient()

// 1. The answer: metadata and Markdown. The only step that runs a browser.
const page = await expand.fetchJson({ url: 'https://example.com' })
if (isFetchOriginError(page) || !page.meta.resources) throw new Error('No Snapshot to read')

// 2. The catalog behind meta.resources: what else was saved. No API key needed.
const snapshotId = page.meta.resources.snapshotId
const catalog = await expand.snapshots.get({ snapshotId })
console.log(catalog.artifacts.map((artifact) 















Letting an agent verify against the page

Main Markdown leaves out the clutter on purpose. Sometimes the thing the task needs went with it: an attribute value, a data- field, a number inside a script tag, a hidden element, a form default. The agent should not fetch the page again. The saved HTML is exact, already paid for, and one GET away.

# 1. The Markdown response names the catalog. No API key for anything below.
MORE=$(sed -n 's/^more: //p' page.md | tr -d '"')
curl -sS "$MORE"

# 2. Look at the first bytes of the original HTML before reading all of it.
curl -sS "https://api.expand.ai/v1/public/snapshots/<snapshot-link-id>/artifacts/html.original/preview?maxBytes=4096" \
  | jq -r .body | base64 -d

# 3. Read the full source and search it.
curl -sS "https://api.expand.ai/v1/public/snapshots/<snapshot-link-id>/artifacts/html.original/content"

If the value was loaded by an XHR rather than rendered into the HTML, list collections.network to find the request, then read its content link for the saved body.

Note
This is a different tool from Highlights. Highlights finds passages that are in the Markdown and State JSON but were too long to read. The HTML is not in that corpus. When the Markdown does not contain the value at all, go to html.original.

Rules for this path:

  • GET the more URL exactly as written, with no auth header. Follow its links, and never guess routes.
  • Preview before content. Read truncated and fall through to content when the preview is cut off or answers 413.
  • Treat a record's url as where the bytes came from. Saved bytes come from the catalog's artifact links and a record's content link, never from the original URL.
  • Never fetch the origin again to fill a gap. Cite the Snapshot the value came from.
Recommendation
Install the Expand Skill with expandai skill install. It already teaches agents the more catalog step.

Next steps

  • Fetching a URL: the fields in the first response.
  • Two ways to highlight: highlight the Markdown and State JSON of a Snapshot you already have.
  • Playground & replay: open the same Snapshot as a person.
  • TypeScript SDK: the snapshots methods.
PreviousPlayground & replay
NextHighlights: Overview

On This Page

Where the trail startsReading a Snapshot from codeLetting an agent verify against the pageNext steps
}
}
}
---
snapshotId: 019c25ee-1a2a-7d6d-9c1b-3f4e5a6b7c8d # the capture ID, for Highlights and the Playground
more: https://api.expand.ai/v1/public/snapshots/ng0EFjewTRF6?format=markdown # the public ID is in the URL
---
collections.assets
The whole capture, as one archivethe wacz artifact
"/v1/public/snapshots/01JPUBLIC01/assets"
,
"artifacts": "/v1/public/snapshots/01JPUBLIC01/artifacts"
},
"artifacts": [
{ "id": "html.original", "kind": "html", "url": ".../artifacts/html.original/content", "preview": ".../artifacts/html.original/preview" },
{ "id": "har", "kind": "har", "url": ".../artifacts/har/content", "preview": ".../artifacts/har/preview" },
{ "id": "wacz", "kind": "wacz", "url": ".../wacz" }
],
"links": {
"markdown": "/v1/public/snapshots/01JPUBLIC01?format=markdown",
"json": "/v1/public/snapshots/01JPUBLIC01"
}
}
=>
artifact.id))
// ['html.original', 'har', 'wacz']
// 3. A collection: every network request the page made.
const network = await expand.snapshots.network({ snapshotId })
// 4. A bounded look at an artifact before downloading all of it.
const preview = await expand.snapshots
.preview({ snapshotId, artifactId: 'html.original', maxBytes: 512 })
.catch(() => undefined)
// 5. The full bytes. Catalog links are paths on the API origin.
const har = catalog.artifacts.find((artifact) => artifact.id === 'har')
if (har) {
const bytes = await fetch(new URL(har.url, page.meta.resources.url)).then((response) => response.arrayBuffer())
}
await expand.close()
\
| grep -o 'data-price="[^"]*"'