Batched Fetch

Start a run

Send all the URLs in one POST. The call returns as soon as the run is accepted. The credit hold and the captures themselves continue in the background.

urls is required and must be non-empty. include is optional, and the only public browserConfig field at start is scrollFullPage. Unknown top-level keys are rejected.

• Every submitted URL uses the same shared settings. The start body does not accept per-URL settings.
• Duplicate URLs are deduped by resolved URL, so the same page is captured once.
• Production callers should authenticate with an API key. Anonymous sessions cannot start a run.

Keep the run ID. Every status check and every page of results is read back through it.

Safe retries

When x-idempotency-key is omitted, the API generates a unique key. Send a stable key when the caller needs to retry a separate request or recover after a process restart. Retrying the same payload with the same explicit key is safe. The API returns a successful response with the same run ID and does not create or charge for another run.

Only reusing the same key with a different request body returns 409 BatchedIdempotencyConflict:

• Keys are scoped to your organization and may be up to 255 characters.
• The API generates a key when omitted. Persist or deterministically reconstruct an explicit key when deduplication must survive separate calls or process restarts.
• Include a logical period in recurring-job keys, such as acme-catalog:2026-07:<digest>.

In the TypeScript SDK:

• When omitted, batched() generates one idempotencyKey and reuses it for the client's configured retry policy.
• Supply a stable idempotencyKey to deduplicate separate calls or calls made after a process restart.
• The same key and payload return the same run ID. Catch ExpandClientApiError with status 409 only for a different payload. Its body contains existingRunId and reason: "payload_mismatch".

Poll results

Read status and results back with the run ID. Page through results with limit and offset.

Keep polling while batchedStatus is QUEUED or RUNNING. Stop once it reaches a terminal status.

• Use a short delay or backoff between polls instead of a tight loop.
• Handle API errors from the poll call the way you handle any request.
• FAILED or CANCELLED means stop polling the run.
• After the run is terminal, inspect each result's status before using it.
• The generated getBatched query params are strings today, so the examples use "10" and "0".

Paging through results

Results come back in pages. Start with offset=0, then keep increasing offset by limit while pagination.hasMore is true.

• limit defaults to 10 and accepts 1 through 100.
• offset defaults to 0 and must be non-negative.
• pagination.total is the number of available result rows for the run.
• Polling only the first page can miss later results.
• Page-level status can differ across pages while the run is still active.

Statuses

batchedStatus describes the whole run. The top-level status describes the current page of results, and each result has its own persisted status.

• QUEUED means the run was accepted but may not be executing yet.
• RUNNING means work is still in progress.
• COMPLETED on batchedStatus means the run is done, not that every URL produced Markdown.
• FAILED and CANCELLED are terminal.
• A page reads COMPLETED only when its rows are stable and requested content extraction succeeded. Requested content can legitimately be empty.
• A page can read FAILED after a completed capture if Snapshot content could not be read or rendered. Successful siblings on that page remain usable.
• Failed extraction outcomes are cached for one minute. Re-poll the same failed page with backoff to retry extraction without starting or charging for another browser capture.
• A page can read COMPLETED while the whole run is still RUNNING. Keep the two fields separate in your code. Do not collapse them into one status.

Response shape

A poll returns the run status, pagination metadata, and a page of per-URL results.

Handle each result independently. A run can finish even when one URL was blocked, failed, or redirected. Use results[].status rather than missing Markdown to identify failed items. A blocked result remains SUCCEEDED and carries data.blocked. Fully populated siblings remain usable. How to read a failed item, and which to re-run, is in error and originError.

If no poll include.markdown is provided, Batched Fetch normalizes results to Markdown by default. For the exact per-result schema, see the API reference.

error and originError

An item carries exactly one of data or error. originError sits outside that pair and can accompany the captured data.

A URL whose origin could not be reached is not a run failure. That covers a DNS failure, broken TLS, a refused connection, or a connection timeout. The item reads SUCCEEDED, because Expand did its job and established that the URL has nothing to serve, and carries a FetchOriginError in error with no data.

An origin that answered with a 404 or 500 did produce a Snapshot, so that item keeps its data and reports the status in originError instead.

Batched Fetch is item-oriented. Treat the run as a container for many Fetch attempts, then inspect each result before using it.

• Blocked target pages can appear as per-item data.blocked, for example blockedType: "botProtection".
• An unreachable origin appears as a per-item error with _tag: "FetchOriginError" and no data. Read its retryable field before scheduling a re-run: false means the URL is dead, not that the run went wrong.
• data.response.originStatusCode helps classify each outcome, for example 403 on a blocked page.
• Pending rows can initially return only minimal data.response.
• A URL-only, unblocked result on a FAILED page can indicate an extraction failure. Missing Markdown on a COMPLETED page can be legitimate empty content.
• Terminal COMPLETED on the run does not override a page-level extraction failure.
• Store per-URL outcome state so a retry collects only the URLs that need it.

What kind and retryable mean, and what is charged, is on Errors & retries.

Supported options

Polling rejects screenshot and appendix include fields with BatchedIncludeUnsupported when the original run did not capture them. New runs always request Markdown and State JSON. Existing runs retain the formats captured when they were created.

Batched Fetch is narrower than single Fetch by design. Use it when many URLs and asynchronous collection matter more than every single Fetch artifact. For the full set of fields and nested defaults, see Fetching a URL.

Concurrency and cancellation

• Batched Fetch queues its child browser captures under the organization's browser concurrency rather than cancelling each other.
• Cancellation is cooperative. A successful cancel request returns CANCELLING while running items drain, or CANCELLED when no work remains. Queued items are cancelled immediately.
• How much runs at once depends on your browser concurrency. See Limits.
• There is no Batched-specific public rate-limit number to plan against today.

Common mistakes

• Polling once, seeing no results, and assuming the run failed.
• Stopping because a page status is COMPLETED while batchedStatus is still RUNNING.
• Ignoring pagination and reading only the first page.
• Assuming COMPLETED means every item has Markdown.
• Re-running URLs whose error reports retryable: false. The origin is unreachable. The run did nothing wrong.
• Requesting screenshot or appendix while polling when the original run did not capture it.
• Looking for a CLI or MCP command for Batched Fetch. There is none today.
• Expecting an automatically generated key to deduplicate separate calls or calls made after a process restart.
• Treating payload_mismatch as a recoverable duplicate instead of fixing key reuse.

Next steps

• Start Batched Fetch API reference: exact create schema.
• Get Batched Status: the exact poll and result schema in the API reference.
• Fetching a URL: the full include surface and limits.
• Browser-rendered pages: capture and scrollFullPage behavior.