The model for browser page Fetches that finds the data behind the page and returns what is useful as State JSON, next to the Main Markdown.
X-Ray finds the data behind pages Fetch opens in a browser. It returns the page's embedded JSON and the JSON its own APIs loaded as State JSON, next to the Main Markdown, so your agent gets the data behind the page from the same call. There is nothing to switch on for a browser page. State JSON is on by default. Files and YouTube videos have no State JSON.
Not everything a page shows is in its HTML. Prices, stock levels, coordinates, and gate numbers often live in JSON the page loads. A Markdown converter never sees that JSON. The usual workaround is driving a full browser for every page, which is slow and expensive.
Each State JSON entry says which. kind is inline or network, and role names the source, such as ld+json, next-data, or network-response.
X-Ray reads the JSON a page carries and keeps the parts a reasonable question could be asked about. It drops analytics, configuration, and state the page only needs to draw itself.
Each State JSON entry carries an evidence id that citations point to, its kind and role, the request url and method for network data, and the data itself in body.
{
"id": 3,
"kind": "network",
"role": "network-response",
"url": "https://www.figma.com/api/plans",
"method": "GET",
"body"
When an entry is too large to send, it comes back without body, with moreTokens and a sample of its top-level keys instead. How to get the rest is on References to more.
| Surface | Where it is |
|---|---|
| JSON object | json, one entry per piece of evidence. meta.stateJson.moreTokens says how much was held back. |
| Markdown text | A ## Expand State JSON section at the end of the document, one fenced json block per entry. stateJson.moreTokens in the frontmatter says how much was held back. |
| URL | ?include=statejson turns State JSON on, and an empty turns it off. It is on by default. |
How to turn State JSON off on each surface is on Include options.
import { ExpandClient, isFetchOriginError } from '@expandai/sdk'
const expand = new ExpandClient()
const page = await expand.fetchJson({ url: 'https://www.figma.com/pricing/' })
if (!isFetchOriginError(page)) {
console.log(page.json) // State JSON
}expandai fetch https://www.figma.com/pricing/ --format json
# json in the output is the State JSON
?include=| Request body | include: { json: false } turns State JSON off when the URL has no ?include=. |
| Batched Fetch | data.json on each successful page result. |
curl https://api.expand.ai/v1/fetch/json \
-H "x-expand-api-key: $EXPAND_API_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://www.figma.com/pricing/"}'