Useful Cheatsheets
Build / AI /

AI Error Handling

Build resilient AI integrations with exponential backoff, rate limit recovery, circuit breakers, and fallback models.

TL;DR

  1. Catch RateLimitError HTTP 429 exceptions and back off with randomized jitter.
  2. Configure exponential-backoff algorithms to survive temporary upstream API outages.
  3. Implement multi-provider model-fallbacks to preserve application uptime during downtime.
Jump to section

Common AI Error Classes

    RateLimitError (429)

    Triggered when exceeding RPM or TPM account limits.

    if (error instanceof OpenAI.RateLimitError) {
  const delaySec = error.headers?.['retry-after'];
  console.warn(`Retry after ${delaySec}s`);
}
    Timeout Errors

    Occurs when network requests fail to complete within target deadline.

    const isTimeout =
  error instanceof OpenAI.APIConnectionTimeoutError;
if (isTimeout) console.warn('AI request timed out');
    BadRequestError (400)

    Permanent validation errors such as invalid schemas or missing fields.

    if (error instanceof OpenAI.BadRequestError) {
  // Do NOT retry; inspect payload parameters
}

Exponential Backoff With Jitter

    Calculate Backoff Delay

    Compute exponential delay with full randomized jitter distribution.

    function getDelay(i: number, base = 500) {
  const exp = Math.min(base * (2 ** i), 10000);
  return Math.floor(Math.random() * exp);
}
    Retry Loop Implementation

    Execute request loop with maximum retry attempt bounds.

    async function retry<T>(
  fn: () => Promise<T>
): Promise<T> {
  for (let i = 0; i < 3; i++) {
    try { return await fn(); }
    catch (err) {
      if (i === 2 || !isRetryable(err)) throw err;
      const ms = getDelay(i);
      await new Promise(r => setTimeout(r, ms));
    }
  }
  throw new Error('Retries exhausted');
}
    IsRetryable Check

    Filter error status codes that represent transient network issues.

    function isRetryable(err: any): boolean {
  const s = err.status ?? err.statusCode;
  return s === 429 || (s >= 500 && s < 600);
}

Model Fallbacks

    Provider Fallback Chain

    Fail over to alternate provider model when primary model fails.

    async function callAI(prompt: string) {
  try {
    return await callClaude(
      'claude-3-5-sonnet', prompt
    );
  } catch (err) {
    console.warn('Claude failed, trying OpenAI');
    return await callOpenAI('gpt-4o', prompt);
  }
}
    Tier Degradation

    Degrade to faster model if frontier model suffers capacity outage.

    try {
  return await callModel('gpt-4o', prompt);
} catch (err) {
  return await callModel('gpt-4o-mini', prompt);
}
    Cached Response Fallback

    Serve stale cached completion when all upstream providers are unreachable.

    const fallback = await cache.get(promptHash);
if (fallback) return fallback;

Circuit Breaker Pattern

    Circuit State Machine

    Track CLOSED, OPEN, and HALF_OPEN states based on recent failures.

    enum CircuitState { CLOSED, OPEN, HALF_OPEN }
let state = CircuitState.CLOSED;
let consecutiveFailures = 0;
    Fast Failure In Open State

    Reject incoming calls immediately without hitting network when OPEN.

    if (state === CircuitState.OPEN) {
  if (Date.now() - lastFailureTime > 30000) {
    state = CircuitState.HALF_OPEN;
  } else {
    throw new Error('Circuit open: service down');
  }
}
    State Reset On Success

    Reset failure counters when probe request succeeds in HALF_OPEN state.

    function recordSuccess() {
  consecutiveFailures = 0;
  state = CircuitState.CLOSED;
}

Tips

  1. Inspect the retry-after header on HTTP 429 responses to delay the subsequent attempt by the exact duration requested by the provider.
  2. Wrap AI client requests in circuit-breakers to fail fast when error rates exceed acceptable operational thresholds.

Warnings

  1. Never retry non-idempotent or non-retryable errors like 401 Unauthorized or 400 BadRequest because they will never succeed without code changes.
  2. Avoid retrying immediately without random-jitter because thousands of concurrent workers retrying at identical intervals create severe thundering herds.

In Practice

FAQ