Useful Cheatsheets
Build / AI / AI Development

Error Handling & Retries

Build resilient AI integrations with exponential backoff, rate limit recovery, circuit breakers, and fallback models.

TL;DR

  1. Catch RateLimitError and retry with exponential backoff plus random jitter.
  2. Retry only temporary failures such as 429 and 5xx responses.
  3. Keep a fallback model ready so outages do not stop your app.
Jump to section

Common AI Error Classes

    RateLimitError (429)

    Triggered when exceeding RPM or TPM account limits.

    if (error instanceof OpenAI.RateLimitError) {
  const delaySec = error.headers?.['retry-after'];
  console.warn(`Retry after ${delaySec}s`);
}
    Timeout Errors

    Occurs when network requests fail to complete within target deadline.

    const isTimeout =
  error instanceof OpenAI.APIConnectionTimeoutError;
if (isTimeout) console.warn('AI request timed out');
    BadRequestError (400)

    Permanent validation errors such as invalid schemas or missing fields.

    if (error instanceof OpenAI.BadRequestError) {
  // Do NOT retry; inspect payload parameters
}

Exponential Backoff With Jitter

    Calculate Backoff Delay

    Compute exponential delay with full randomized jitter distribution.

    function getDelay(i: number, base = 500) {
  const exp = Math.min(base * (2 ** i), 10000);
  return Math.floor(Math.random() * exp);
}
    Retry Loop Implementation

    Execute request loop with maximum retry attempt bounds.

    async function retry<T>(
  fn: () => Promise<T>
): Promise<T> {
  for (let i = 0; i < 3; i++) {
    try { return await fn(); }
    catch (err) {
      if (i === 2 || !isRetryable(err)) throw err;
      const ms = getDelay(i);
      await new Promise(r => setTimeout(r, ms));
    }
  }
  throw new Error('Retries exhausted');
}
    IsRetryable Check

    Filter error status codes that represent transient network issues.

    function isRetryable(err: any): boolean {
  const s = err.status ?? err.statusCode;
  return s === 429 || (s >= 500 && s < 600);
}

Model Fallbacks

    Provider Fallback Chain

    Fail over to alternate provider model when primary model fails.

    async function callAI(prompt: string) {
  try {
    return await callClaude(
      'claude-sonnet-5-5', prompt
    );
  } catch (err) {
    console.warn('Claude failed, trying OpenAI');
    return await callOpenAI('gpt-6.1-sol', prompt);
  }
}
    Tier Degradation

    Degrade to faster model if frontier model suffers capacity outage.

    try {
  return await callModel('gpt-6.1-sol', prompt);
} catch (err) {
  return await callModel('gpt-6-luna', prompt);
}
    Cached Response Fallback

    Serve stale cached completion when all upstream providers are unreachable.

    const fallback = await cache.get(promptHash);
if (fallback) return fallback;

Circuit Breaker Pattern

    Circuit State Machine

    Track CLOSED, OPEN, and HALF_OPEN states based on recent failures.

    enum CircuitState { CLOSED, OPEN, HALF_OPEN }
let state = CircuitState.CLOSED;
let consecutiveFailures = 0;
    Fast Failure In Open State

    Reject incoming calls immediately without hitting network when OPEN.

    if (state === CircuitState.OPEN) {
  if (Date.now() - lastFailureTime > 30000) {
    state = CircuitState.HALF_OPEN;
  } else {
    throw new Error('Circuit open: service down');
  }
}
    State Reset On Success

    Reset failure counters when probe request succeeds in HALF_OPEN state.

    function recordSuccess() {
  consecutiveFailures = 0;
  state = CircuitState.CLOSED;
}

Tips

  1. Read the retry-after header on HTTP 429 responses and wait exactly as long as the provider asks before retrying.
  2. Wrap requests in a circuitBreaker so your app fails fast when the provider error rate climbs past a threshold.

Warnings

  1. Never retry non-idempotent or non-retryable errors like 401 Unauthorized or 400 BadRequest because they will never succeed without code changes.
  2. Avoid retrying on a fixed schedule without Math.random() jitter, because many workers retrying together create a thundering herd.

In Practice

FAQ