The two modes, what to include in a Fetch, and the pointers to everything held back.
Fetch has two modes. Markdown Mode returns Markdown text, for agents. JSON Mode returns a JSON object, for applications. Both carry the same layers. The Markdown text puts the metadata in YAML frontmatter at the top, and the JSON object puts each layer in its own field.
Markdown Mode is the default. It gives your agent the Main Markdown, enough to work from without flooding its context, and suits first-pass reading, summarization, extraction, and browsing. Include more layers only when the task needs more context or a different artifact. See Progressive disclosure.
The same Fetch in both modes, with every layer included.
---
url: news.ycombinator.com
title: Hacker News
playground: expand.land/s/...
---# Hacker News
1. Example story title
2. Another story title## Appendix
- [new](/newest) · [past](/front)## Expand State JSON
```json
{ "stories": [ ... ] }
```"meta": {
"url": "news.ycombinator.com",
"snapshotId": "...",
"appendix": { "url": "..." }
},"markdown":
"# Hacker News\n\n1. Example…Pick Markdown Mode when the response goes straight to an agent or a prompt. The Markdown text is one string, ready to read.
Pick JSON Mode when your code reads the fields separately, such as the snapshotId or the State JSON.
/pricing, or #faq for the page itself. Resolve them against finalUrl, or url when there is no finalUrl. Links to other sites stay absolute.The Main Markdown always comes back whole, however long the page. Only a Highlights request returns less. Citations point back to the page through evidence IDs: json[].id on State JSON entries, and location.evidenceId on Highlights results, which come back in data.highlights.
You choose what a Fetch returns in two places, in either mode:
?include= parameter turns two layers on and off: State JSON and the Appendix. An empty ?include= turns State JSON off. ?include=appendix,statejson turns both on.include.appendix and include.json do the same when the URL has no ?include=. Without either, you get State JSON but not the Appendix.include field asks for extra artifacts outside the document, such as a screenshot. Only JSON Mode can return one. In Markdown Mode, a requested screenshot is saved with the Snapshot but not returned.Pick a format and what to include. The code below shows how to ask for it on each surface.
import { ExpandClient } from '@expandai/sdk'
const expand = new ExpandClient()
const markdown = await expand.fetch({ url: 'https://news.ycombinator.com' })expandai fetch https://news.ycombinator.com
# The Appendix, afterwards
expandai appendix <snapshotId>// Arguments for the fetch tool
{
"url": "https://news.ycombinator.com"
}curl https://api.expand.ai/v1/fetch \
-H "x-expand-api-key: $EXPAND_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://news.ycombinator.com"
}'Response
---
url: https://news.ycombinator.com
snapshotId: ...
playground: https://expand.land/s/...
title: Hacker News
description: ...
appendix:
url: https://expand.land/s/.../appendix
links:
sameDomain: 42
external: 7
stateJson:
moreTokens: 1200 # only when some State JSON was held back
---
# Hacker News
[appendix omitted: 14 links · ~320 tokens] {612}
1. Example story title
2. Another story title
## Expand State JSON
```json id=3
{ "stories": [ ... ] }
```Not every field is shown. The full response is in the API reference.
Rules to remember:
statejson in URLs. Other spellings, such as stateJson and state-json, also work.?include=appendix on its own turns State JSON off. List both to keep both.?include=, even an empty one, it wins over the body's include.appendix and include.json.include too, but only for what the run captured.To keep responses small, Fetch sends a pointer instead of content that's large or secondary. Each pointer says what was held back and where to get it, so nothing is lost.
| Pointer | What was held back | How to get it |
|---|---|---|
appendix in the frontmatter, meta.appendix in the JSON object | The Appendix | Open its url, or fetch with ?include=appendix |
[appendix omitted: …] {612} in the Main Markdown | One piece of the Appendix |
Everything else the capture saved stays one request away: meta.resources links to it. See Progressive disclosure.
meta.resources and the artifacts behind a Snapshot. …\n\n## Appendix\n\n- [new]…","json": [
{ "stories": [ ... ] }
]expandai appendix <snapshotId> --id 612stateJson.moreTokens | Some State JSON, about that many tokens | Run Highlights on the Snapshot |
A json entry with moreTokens and keys but no body | That entry, too large to send | Request /s/<snapshotId>?id=<id>, or open it in Playground & replay |
meta.markdown on a Highlights response | The full Main Markdown | Open its url |