Expand AI logo
DocsAPI ReferenceAPI Reference
Glow Active
Login

API Reference

Getting Started

API ReferenceRate LimitingError Handling

Endpoints

Start Batched FetchCancel Batched FetchFetchFetch JSON ModeHighlightsGet Batched Status
Browse docs

Getting Started

API ReferenceRate LimitingError Handling

Endpoints

Start Batched FetchCancel Batched FetchFetchFetch JSON ModeHighlightsGet Batched Status

Error Handling

When Expand fails, when the origin fails, and how to tell the two apart.

Expand separates its own failures from the origin's. That split is the single rule behind every status code and every error body in this API.

The governing rule

A non-2xx status means Expand failed to do its job. A 200 means Expand did its job, and the body reports what the did.

origin

Fetching a URL involves two parties. Expand can fail — infrastructure trouble, a bot wall it could not get past, a page beyond its processing limits. The origin can also fail — DNS that does not resolve, a broken certificate, a refused connection, a 404. A status code alone cannot express whose failure it was, so Expand does not try: origin outcomes are typed data inside a successful response.

Whose failureHow it is reported
Expand'sNon-2xx status with a _tag-discriminated error body
The origin's200 plus a FetchOriginError in the body (or in response headers, on /v1/fetch)

The practical consequence: nothing retries a dead URL by accident. Generic retry middleware retries 502 and 503. It does not retry 200. A permanently dead origin no longer looks like a temporary Expand outage.

Error taxonomy

Every error carries a _tag naming its class, and the classes whose status alone cannot settle the question also carry a retryable verdict. Status and tag together are the contract; the tag is what you should branch on.

Status_tagWhat happenedRetry
200FetchOriginError in the body or response headersExpand completed the request; the origin failedFollow the origin outcome's retryable field
400,

A successful request returns 200. Statuses that carry more than one tag return them as an anyOf in the OpenAPI spec: the status tells you the category, the _tag tells you which member of it.

The fetch failure path is a three-way partition. Proven origin failures are 200 outcomes. Proven deterministic page or capture failures are 502 with retryable: false. Failures that fit neither proof stay in the smaller 503 residue with retryable: true; a missing or unreviewed network code is not enough to blame the origin or the page.

Every response — success or error — carries the x-expand-trace-id header. Quote it when contacting support.

Error response format

Every error body carries a _tag field identifying the error type:

{
  "_tag": "ErrorType"
  // additional fields depend on the error type
}

Origin outcomes (200)

When the origin — not Expand — is the thing that failed, the response is a 200 carrying a FetchOriginError.

{
  "_tag": "FetchOriginError",
  "url": "https://checkout.deadstartup.example/pricing",
  "kind": "dns",
  "upstreamError": "net::ERR_NAME_NOT_RESOLVED",
  "retryable": false
}
FieldDescription
urlThe requested URL this outcome describes.
kindWhat went wrong at the origin. See the table below.
upstreamErrorThe underlying browser network error, for the network kinds. Optional.
originStatusCode

Kinds

kindMeaningretryable
dnsDNS resolution failed. The exact code distinguishes a stable missing name from resolver timeouts/failures.Depends on upstreamError.
tlsThe TLS handshake or certificate validation failed.Depends on upstreamError.

The network kinds mean no HTTP response ever existed. http means the origin answered and told you it had a problem.

For current captures, retryable is derived from the exact reviewed Chromium net::ERR_* code, not merely from kind. For example, net::ERR_NAME_NOT_RESOLVED is a stable DNS failure and reports false, while net::ERR_DNS_TIMED_OUT is transient and reports true. Unknown codes from current native capture stay Expand-side failures rather than receiving a guessed origin kind or retry verdict. Older stored failures may not carry a reviewed code; when those already contain a legacy failureKind, the API preserves that kind and uses its conservative default retry verdict.

For kind: "http", retryable is true when originStatusCode is 408, 429, or 500 and above — the origin either asked for a later attempt or hit a server-side condition that may clear. Every other 4xx describes the request itself, so repeating it unchanged reports false.

POST /v1/fetch/json

The JSON endpoint returns one of two shapes, and they are disjoint.

Origin never reached — no document exists, so the origin outcome is the response. Recognize this variant by its top-level _tag:

HTTP/1.1 200 OK

{
  "_tag": "FetchOriginError",
  "url": "https://checkout.deadstartup.example/pricing",
  "kind": "dns",
  "upstreamError": "net::ERR_NAME_NOT_RESOLVED",
  "retryable": false

Origin responded with an error status — the page was captured, so you get the content and the outcome. Error pages are frequently useful: soft-404s, maintenance notices, status dashboards returning 500. Recognize the content variant by its meta field; it has no top-level _tag.

HTTP/1.1 200 OK

{
  "meta": {
    "version": 1,
    "url": "https://site.example/removed-page",
    "title": "Page not found"
  },
  "markdown": "# Page not found\n\nThe page you are looking for has been removed…",
  "

originError is absent whenever the origin answered normally, so its presence alone tells you the origin reported a problem.

POST /v1/fetch

/v1/fetch answers in text/markdown, which has nowhere to put a typed field, so the origin outcome rides response headers.

HeaderValue
x-expand-origin-errorThe kind — dns, tls, connectionRefused, connectionReset, connectionClosed, addressUnreachable, emptyResponse, connectionTimeout, or http.
x-expand-origin-error-detail

When the origin was never reached there is no document to return, so the body is a short Markdown error document written to be read by an LLM as readily as by a human:

HTTP/1.1 200 OK
content-type: text/markdown; charset=utf-8
x-expand-origin-error: dns
x-expand-origin-error-detail: net::ERR_NAME_NOT_RESOLVED
x-expand-origin-error-retryable: false

# Origin unreachable

DNS resolution failed for `https://checkout.deadstartup.example/pricing`
(`net::ERR_NAME_NOT_RESOLVED`). No HTTP response exists at this URL. Retrying
will not succeed until the domain's DNS is fixed.

When the origin responded with an error status, the captured content is returned as usual and only the headers change:

HTTP/1.1 200 OK
content-type: text/markdown; charset=utf-8
x-expand-origin-error: http
x-expand-origin-error-detail: 500
x-expand-origin-error-retryable: true

---
url: https://api-status.example/dashboard
title: Internal Server Error
---
# Internal Server Error
…captured page content…

The headers are absent entirely when the origin answered normally.

The high-level SDK fetch convenience returns the Markdown string. Use the SDK's raw-response surface where available, or a direct HTTP request, when you also need to inspect these response headers.

Batched Fetch

Batched Fetch reports the same type per URL, in-band. A run never fails because one URL was unreachable.

{
  "status": "SUCCEEDED",
  "error": {
    "_tag": "FetchOriginError",
    "url": "https://checkout.deadstartup.example/pricing",
    "kind": "dns",
    "upstreamError

An item carries exactly one of data or error: error means no snapshot was produced. An origin that responded with an error status did produce one, so that item keeps its data and reports the outcome in a separate originError field alongside it. See Batched Fetch.

Expand's own errors (non-2xx)

retryable

A status code cannot always tell you whether a retry could succeed: a 502 here means the page itself could not be captured and will fail the same way next time, while a 503 means our side had a transient problem. Every error whose status leaves that open — 429, 500, 502, 503 — therefore states the answer outright:

{
  "_tag": "ServiceUnavailable",
  "retryable": true
}

Prefer this field over the status code. Errors that are unambiguous from their status alone (400, 401, 403, 404, 409, 413) omit it; treat a missing field on those as false.

429 and 503 responses also carry a Retry-After header, in seconds. Wait at least that long before retrying — retrying sooner spends your own timeout budget on a refusal we already predicted.

Error types

Validation errors (400)

Returned when the request body does not match the expected schema. Framework-level decoding produces HttpApiDecodeError:

{
  "_tag": "HttpApiDecodeError",
  "message": "Invalid request parameters",
  "issues": [
    {
      "_tag": "Missing",
      "path": ["url"],
      "

Endpoint-level validation — unknown properties, or a search request with no searchable corpus — produces BadRequest:

{
  "_tag": "BadRequest",
  "message": "Invalid fetch request body"
}

Common validation issues:

  • Missing required url field
  • Invalid URL format (must be http:// or https://)
  • Unknown top-level keys in the request body
  • Invalid regex pattern in include.links.includePatterns
  • Empty search.query

Authentication errors (401)

{
  "_tag": "AuthFailed",
  "reason": "InvalidApiKey",
  "description": "The provided API key is not valid"
}
ReasonDescription
InvalidApiKeyThe API key is missing or invalid
InvalidTokenThe bearer token is invalid
InvalidSessionThe session has expired
InvalidTenant

Access and blocked errors (403)

AccessBlocked means your organization cannot currently run fetches — most often a zero credit balance. See Pricing & Usage.

{
  "_tag": "AccessBlocked",
  "organizationId": "org_123"
}

FetchBlocked means the target site blocked the capture and Expand could not get past it. This is Expand failing to deliver the page, which is why it is a 403 rather than an origin outcome.

{
  "_tag": "FetchBlocked",
  "url": "https://example.com",
  "blockedType": "botProtection"
}

blockedType is optional and describes the classification, for example captcha, botProtection, ipBlock, or authWall. Retrying immediately rarely helps; Expand already retried through a different proxy exit where that could plausibly clear the block.

Size errors (413)

PayloadTooLarge is about your request. FetchPageTooLarge is about the target page: the request was well-formed, the page is simply beyond what Expand will process.

{
  "_tag": "FetchPageTooLarge",
  "url": "https://example.com/huge",
  "limitKind": "domsnapshot_bytes",
  "limitBytes": 52428800,
  "measuredBytes": 91234567
}

limitKind is one of transfer_bytes, domsnapshot_bytes, or ast_bytes. Retrying the same URL will not help.

Navigation errors (502)

FetchNavigationFailed means navigation committed no HTTP(S) document and no recognized Chromium network code proved the origin was responsible. Browser documents such as about:blank, blob:, data:, extension/PDF viewers, and download outcomes are not evidence of a refused origin connection, so they remain Expand-side failures.

{
  "_tag": "FetchNavigationFailed",
  "url": "https://example.com/report.pdf",
  "failureType": "browserInternalDocument"
}

This also preserves the honest response for legacy batched results that stored only navigationFailed without a network code.

Rate limit errors (429)

Returned when you exceed your rate limit. See Rate Limiting.

{
  "_tag": "TooManyRequests",
  "retryable": true
}

Capacity errors (429)

Returned when your organization already has as many captures in flight as your plan allows. Nothing was captured, so the same request succeeds once a slot frees:

{
  "_tag": "FetchConcurrencyLimitExceeded",
  "maxConcurrency": 5,
  "retryAfterSeconds": 20,
  "requestId": "0f4c...",
  "retryable": true
}

maxConcurrency is your own limit, not a global one. Either lower the number of requests you run at once, or send the URLs to /v1/fetch/batched, which queues against the same limit instead of refusing.

Internal errors (500)

Returned when an unexpected server-side failure interrupts the request:

{
  "_tag": "InternalError",
  "retryable": true
}

An unexpected failure may not recur, so a retry is worthwhile.

Page Capture Errors (502)

Returned when the browser could not produce a usable page for the URL. Two tags share this status.

FetchNavigationFailed — navigation itself did not reach the page:

{
  "_tag": "FetchNavigationFailed",
  "url": "https://example.com/",
  "failureType": "browserInternalDocument",
  "retryable": false
}

FetchCaptureFailed — navigation succeeded, but the capture did not:

{
  "_tag": "FetchCaptureFailed",
  "url": "https://example.com/",
  "failureType": "pageNeverSettled",
  "retryable": false
}
failureTypeMeaning
pageNeverSettledThe page never produced any navigation activity to capture
capabilityFailedThe page rendered, but no capture satisfied a format you requested

Both verdicts are deterministic for this URL: the captured evidence proves another attempt will reach the same outcome. Fix the URL or the target page instead.

The SDKs do not retry the page verdicts or FetchCaptureTimeout. 403 and 413 are not retryable statuses, and the 502 and 504 tags are explicitly exempted. Keeping ambiguous 504s out of automatic retry loops avoids amplifying a control-plane incident; a deliberate manual retry may still be appropriate. The SDKs do retry 529 FetchCapacityTimeout with their normal backoff; the response also advertises Retry-After for direct HTTP clients.

Capture timeouts (504)

Returned when no publishable result reached the API inside the server-side request deadline and Expand could not prove the run remained queue-only — captures known to have started, plus the conservative fallback when classification is missing, fails, or exceeds its five-second budget:

{
  "_tag": "FetchCaptureTimeout",
  "url": "https://example.com/",
  "timeoutMs": 150000,
  "retryable": false
}
  • Synchronous Fetch uses one 150-second capture-work budget created before Hatchet dispatch. Dispatch, retries, worker reassignments, browser work, and result publication all share its immutable epoch. At expiry, timeout classification has one additional five-second budget and is disconnected from a hung control-plane call. A result that arrives during classification can still win and be returned successfully; otherwise a typed timeout is produced by 155 seconds at the latest.
  • timeoutMs reports the configured request ceiling (150000 by default), not a remaining worker budget.
  • retryable: false is why the SDKs do not retry this automatically: it keeps ambiguous 504s out of automatic retry loops, so a control-plane incident is not amplified. It is not proof the page was at fault, and a deliberate manual retry may still be appropriate — it starts a separate capture that can consume another full 150-second budget.
  • Use a synchronous client timeout of 180000 ms. If timeout classification wins, the server produces a typed timeout by 155 seconds, leaving 25 seconds for response serialization and network transit. The TypeScript SDK's timeoutMs default of 60000 aborts long captures client-side. See TypeScript SDK.
  • Batched items receive their epoch at browser-child dispatch. A child that first starts after expiry persists FetchCapacityTimeout; a replacement for a child that started earlier stays on the 504 side.

Capacity timeouts (529)

Returned when Expand proves the request remained queue-only until the request deadline:

{
  "_tag": "FetchCapacityTimeout",
  "url": "https://example.com/",
  "timeoutMs": 150000,
  "retryable": true
}
  • This is Expand capacity exhaustion, not a verdict about the target page: nothing ran, so the same request can succeed once a slot frees. That is what retryable: true says, and it is the one timeout the SDKs retry for you.
  • Responses carry Retry-After: 5 — a short overload backoff that avoids an immediate retry stampede while spending little of the next request's 150-second budget. Direct HTTP clients should wait at least that long.
  • If a worker started and later disappeared, the request stays 504 FetchCaptureTimeout: the historical start proves it was not queue-only.

Service errors (503)

ServiceUnavailable now means only one thing: an Expand-side infrastructure problem. It is no longer returned for unreachable origins.

{
  "_tag": "ServiceUnavailable",
  "message": "Expand could not complete the request. Retry shortly, or contact support with the request ID if the problem persists.",
  "requestId": "0f4c...",
  "retryable": true
}

This is reserved for transient failures on our side. Retry after the delay named by Retry-After, and quote requestId if the problem persists. requestId is the same trace ID sent in the x-expand-trace-id header.

When to retry

OutcomeRetry?
200 with FetchOriginError, retryable: trueYes, after a delay. The condition may clear.
200 with FetchOriginError, retryable: falseNo. The URL is dead; fix the URL or drop it.
200 with no origin errorNothing to retry.

Do not treat a 200 carrying an origin error as a transport failure. It is a completed answer about a URL that could not serve content.

Billing by outcome

You pay for delivered content, not for attempts. Charges follow the outcome, not the status code.

OutcomeBilled
Page captured successfullyYes
Origin responded with an error status (400+) and content was capturedYes — the content was delivered alongside the originError
Origin unreachable — any network FetchOriginError kindNo
FetchBlocked (403)

The one case that surprises people is the second row: an origin's 404 or 500 page still contains content — soft-404s, maintenance pages, status dashboards — and Expand captured and returned it, so it is billed like any other page. If you do not want to pay for error pages, check originError before treating a response as useful.

Everything else is straightforward: if no content reached you, no charge was made. See Pricing & Usage for how billable outputs are metered.

Handling errors

With the TypeScript SDK

ExpandClient throws only when Expand failed. Inspect status and body on ExpandClientApiError to branch on the _tag. See the TypeScript SDK for the full error hierarchy.

import { ExpandClient, ExpandClientApiError, ExpandClientError } from '@expandai/sdk'

const expand = new ExpandClient({ apiKey: '{{API_KEY}}' })

try {
  const result = await expand.fetchJson({ url: 'https://example.com' })
  // Reached here means Expand succeeded.
} catch (error) {
  if (error instanceof ExpandClientApiError) {
    // A non-2xx response. `body` is the parsed error above — branch on `_tag`.
    console.log('Status:', error.status)

fetchJson resolves to either shape, so narrow before reading content. Only the origin-outcome variant carries a top-level _tag:

import { isFetchOriginError } from '@expandai/sdk'

const result = await expand.fetchJson({ url: 'https://example.com' })

if (isFetchOriginError(result)) {
  // The origin was never reached — there is no content.
  console.log(result.kind, result.retryable)
} else if (result.originError) {
  // The origin answered with an error status, but the page was captured.
  console.log(result.originError.originStatusCode, result.markdown)
} else {
  console.log(result.markdown)
}

With cURL

Check the status code first, then the body. On /v1/fetch, the origin outcome is in the response headers.

response=$(curl -s -D /tmp/expand-headers -w "\n%{http_code}" \
  -X POST https://api.expand.ai/v1/fetch \
  -H "x-expand-api-key: {{API_KEY}}" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}')

status_code=$(echo "$response" | tail -n 1)
body=$(echo "$response

Best practices

  1. Check the status first, then the body. A non-2xx is Expand's problem; a 200 may still carry an origin outcome.
  2. Branch on _tag, not on message text.
  3. Trust retryable. It is derived from the exact Chromium network code; false means stop, while true means the condition may clear.
  4. Retry only outcomes marked retryable: true, with backoff and never sooner than Retry-After.
  5. Do not automatically retry page verdicts (403, 413, 502) or the conservative timeout fallback (504). A deliberate 504 retry may still be appropriate, but each attempt re-runs a full capture.
  6. Use a 180000 ms synchronous client timeout: after the 150-second capture-work budget, bounded classification produces a typed timeout by 155 seconds at the latest if no result wins first, leaving 25 seconds for delivery.
  7. Record x-expand-trace-id on every response. It is what support needs, and it is present even when the body is not.

On This Page

The governing ruleError taxonomyError response formatOrigin outcomes (200)KindsPOST /v1/fetch/jsonPOST /v1/fetchBatched FetchExpand's own errors (non-2xx)retryableError typesValidation errors (400)Authentication errors (401)Access and blocked errors (403)Size errors (413)Navigation errors (502)Rate limit errors (429)Capacity errors (429)Internal errors (500)Page Capture Errors (502)Capture timeouts (504)Capacity timeouts (529)Service errors (503)When to retryBilling by outcomeHandling errorsWith the TypeScript SDKWith cURLBest practices
HttpApiDecodeError
BadRequest
The request body did not match the schema
Never
401AuthFailedThe API key, token, or session is not validNever
403AccessBlockedYour organization's access is suspendedNever
403FetchBlockedThe target site refused the capture (bot protection, captcha)Never
404FetchSnapshotSearchNotFoundNo snapshot exists to searchNever
409BatchedIdempotencyConflict, BatchedCancellationConflictThe batched run is in a conflicting stateNever
413PayloadTooLargeThe request body exceeds the limitNever
413FetchPageTooLargeThe captured page exceeds the limitNever
429TooManyRequestsYou exceeded your rate limitretryable: true
429FetchConcurrencyLimitExceededYou already have as many captures running as your plan allowsretryable: true
500InternalErrorAn unexpected server-side failure interrupted the requestretryable: true
502FetchNavigationFailedThe browser could not navigate to the pageretryable: false
502FetchCaptureFailedThe page could not be capturedretryable: false
503ServiceUnavailableA transient failure on our sideretryable: true
504FetchCaptureTimeoutThe capture ran but produced no publishable result before its deadlineretryable: false
529FetchCapacityTimeoutThe request expired in the queue; the capture never startedretryable: true
The origin's HTTP status. Present only for kind: "http".
retryableWhether retrying the same URL later could plausibly succeed.
connectionRefusedThe connection attempt was explicitly refused; nothing accepted it.false
connectionResetA peer or intermediary reset an established connection (TCP RST).true
connectionClosedA peer closed an established connection unexpectedly (TCP FIN).true
addressUnreachableThe network had no working route to the origin address.true
emptyResponseThe server accepted the connection, then closed it without sending data.true
connectionTimeoutThe connection never answered in time.true
httpThe origin responded, with a status of 400 or above.Depends on the status — see below.
}
json
"
: [],
"originError": {
"_tag": "FetchOriginError",
"url": "https://site.example/removed-page",
"kind": "http",
"originStatusCode": 404,
"retryable": false
}
}
The net::ERR_* code for the network kinds, or the origin's status for http. Omitted when neither is known.
x-expand-origin-error-retryabletrue or false.
"
:
"net::ERR_NAME_NOT_RESOLVED"
,
"retryable": false
}
}
message
"
:
"is missing"
}
]
}
The organization was not found
400, 401, 403, 413No. Fix the request, the credentials, the balance, or the URL.
429Yes, following retryable and Retry-After. See Rate Limiting.
500Yes when the body says retryable: true; report it with the trace ID if it persists.
502 with FetchNavigationFailed or FetchCaptureFailedNo. The page or capture failure was classified as deterministic.
503Yes, after Retry-After; this is the unproven Expand-side residue.
504 with FetchCaptureTimeoutNo automatic retry. A deliberate retry may still be appropriate.
529 with FetchCapacityTimeoutYes, after Retry-After; the capture never started.
No
FetchPageTooLarge (413)No
ServiceUnavailable (503) or InternalError (500)No
console.log(
'Tag:'
, (error.body
as
{ _tag?: string })?._tag)
} else if (error instanceof ExpandClientError) {
// Everything else the SDK can throw: timeouts, connection failures, bad options.
console.log('Error:', error.message)
}
}

The SDK already retries for you, using the same retryable field: it never retries a 502 FetchCaptureFailed, and it waits out Retry-After before retrying a 429. Anything that reaches your catch block has run out of attempts.

"
| sed
'$d'
)
if [ "$status_code" -ne 200 ]; then
# Expand failed. The body carries a _tag.
echo "Expand error ($status_code): $body"
elif grep -qi '^x-expand-origin-error:' /tmp/expand-headers; then
# Expand succeeded; the origin did not.
grep -i '^x-expand-origin-error' /tmp/expand-headers
fi
  • Validate inputs before sending to avoid 400 errors.