Expand AI logo
DocsAPI ReferenceAPI Reference
Glow Active
Login

API Reference

Getting Started

API ReferenceRate LimitingError Handling

Endpoints

Start Batched FetchCancel Batched FetchFetchFetch JSON ModeHighlightsGet Batched Status
Browse docs

Getting Started

API ReferenceRate LimitingError Handling

Endpoints

Start Batched FetchCancel Batched FetchFetchFetch JSON ModeHighlightsGet Batched Status

Rate Limiting

API rate limits and how to handle them

The Expand API rate-limits requests to keep usage fair across organizations.

How rate limiting works

Rate limits apply per organization. When you exceed your rate limit, the API returns a 429 Too Many Requests status.

Rate limit response

When you are rate limited, the API returns a 429 status with the error tag TooManyRequests and a Retry-After header (in seconds):

{ "_tag": "TooManyRequests", "retryable": true }

retryable is always true here — the same request succeeds once the limit resets. Wait at least as long as Retry-After asks before retrying. See Error Handling for the field's full contract.

Handling rate limits

With the TypeScript SDK

The SDK retries rate limits for you: a 429 declares itself retryable, and the SDK waits out Retry-After before trying again. You only need to handle the case where it runs out of attempts:

import { ExpandClient, ExpandClientApiError } from '@expandai/sdk'

const client = new ExpandClient({ apiKey: '{{API_KEY}}', maxRetries: 5 })

try {
  const result = await client.fetch({ url: 'https://example.com' })
} catch (error) {
  if (error instanceof ExpandClientApiError && error.status === 429) {
    // Still rate limited after every retry — back off at the application level.
    console.log('Rate limited:', (error.body as { _tag?: string })?._tag)

Retry strategy

If you are calling the API directly rather than through an SDK, retry only when the body says to, and never sooner than Retry-After asks:

  1. Read retryable from the error body. If it is false, stop — no retry can succeed
  2. Wait at least Retry-After seconds (or the body's retryAfterSeconds, if present)
  3. Back off exponentially from there, up to a maximum (e.g., 32 seconds)
  4. Jitter the wait. Every client rejected in the same burst is handed the same Retry-After, so an unjittered wait sends the whole population back at one instant
  5. After your maximum attempts, surface the error to the user

A successful /v1/fetch responds with text/markdown, not JSON — read it with response.text(). Only the error responses are JSON, and a failure from an intermediary (a proxy's own error page, an empty body) may not be JSON either, so parsing it has to be allowed to fail.

async function fetchWithRetry(url: string, maxRetries = 5) {
  let delay = 1000

  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const response = await fetch('https://api.expand.ai/v1/fetch', {
      method: 'POST',
      headers: { 'x-expand-api-key': API_KEY, 'content-type': 'application/json' },
      body: JSON.stringify({ url }),
    })
    // The success body is the Markdown document itself.

Best practices

  • Batch requests where you can to cut down on API calls.
  • Cache responses instead of re-fetching the same URL.
  • Watch your usage in the dashboard.
  • Retry with exponential backoff to ride out transient limits.

Need higher limits?

If the default limits are too low for you, email support@expand.ai about enterprise options.

On This Page

How rate limiting worksRate limit responseHandling rate limitsWith the TypeScript SDKRetry strategyBest practicesNeed higher limits?
}
}
if (response.ok) return response.text()
// An undecodable error body tells you nothing about retryability. Fall back to
// the status: 408/429/5xx are the transient ones. 409 is a conflict verdict,
// not a transient failure — retrying reaches the same answer.
const body = await response.json().catch(() => undefined)
const retryable =
typeof body?.retryable === 'boolean'
? body.retryable
: response.status === 408 || response.status === 429 || response.status >= 500
if (!retryable || attempt === maxRetries - 1) throw new Error(body?._tag ?? `HTTP ${response.status}`)
// The body wins over the header: it survives proxies that strip headers, and it is the
// value the server chose for this error rather than the default for this status.
const retryAfter =
Number(body?.retryAfterSeconds ?? response.headers.get('retry-after') ?? 0) * 1000
const wait = Math.max(delay, retryAfter)
// Spread over [1x, 1.25x] — upward only, since `Retry-After` names the earliest
// moment worth trying again and jittering below it retries sooner than asked.
await new Promise(resolve => setTimeout(resolve, wait * (1 + Math.random() * 0.25)))
delay = Math.min(delay * 2, 32_000)
}
}