What does a 429 from an AI API mean?

You have hit a rate limit: too many requests or tokens in a window. The response often includes a Retry-After header telling you how many seconds to wait. Back off, wait, and retry rather than failing immediately.

What is exponential backoff with jitter?

You wait longer after each failed attempt, doubling the delay, and add a small random offset. Doubling avoids hammering the API; the random jitter stops many clients retrying in lockstep and causing another spike.

How do I avoid rate limits in the first place?

Limit how many requests you send at once, cache and dedupe repeats, and debounce input. A small client-side concurrency limit plus server-side per-user limits keeps you comfortably under the provider's ceiling.

Build / Code / React

Handling AI Rate Limits

Recover from AI rate limits by respecting Retry-After and retrying with exponential backoff and jitter.

TL;DR

Detect a 429 status and read the Retry-After header.
Retry with exponential backoff plus random jitter.
Cap retries and limit concurrency to avoid a storm.

Jump to section

Detect the Limit
Back Off and Retry
Limit Concurrency
Tell the User
Tips
Warnings
In Practice
FAQ

Detect the Limit

Check the Status

A 429 means you are being rate limited.

if (res.status === 429) await backoff();

Read Retry-After

The header says how long to wait.

const ra = res.headers.get('Retry-After');

Parse Seconds

Convert the header to milliseconds.

const ms = Number(ra) * 1000;

Back Off and Retry

Exponential Delay

Double the wait after each attempt.

const delay = base * 2 ** attempt;

Add Jitter

Randomize so clients do not sync up.

const wait = delay + Math.random() * 250;

Cap Attempts

Give up after a few tries.

if (attempt >= 5)
  throw new Error('rate limited');

Limit Concurrency

Max In Flight

Only allow a few requests at once.

if (active >= 3) await queue.next();

Queue Extras

Hold the rest until a slot frees.

queue.push(task);

Per-User Cap

Enforce a limit on the server too.

// server: N requests per user / minute

Tell the User

Show a Wait

Let them know a retry is coming.

setStatus('Busy, retrying...');

Disable Send

Block new requests while retrying.

<button disabled={retrying}>Send</button>

Final Error

Surface a clear message if it gives up.

setError('Too many requests, try later');

Tips

Honor the Retry-After header when present; it tells you exactly how long to wait, which beats guessing with backoff alone.
Add random jitter to each backoff delay so many clients do not retry in sync and hammer the API at the same instant.

Warnings

Do not retry forever; cap attempts and surface a clear error, or a rate-limited user waits behind endless silent retries.
Retrying a non-idempotent action can double an effect; only auto-retry safe reads, not writes with side effects.

In Practice

A fetch wrapper that retries 429s with exponential backoff, jitter, and an attempt cap.

On a 429, it reads Retry-After or falls back to exponential backoff.
Each delay doubles and adds jitter so retries spread out.
It gives up after a fixed number of attempts and throws.
Any non-429 response returns immediately.

async function askWithRetry(body: unknown, tries = 4) {
  for (let attempt = 0; attempt < tries; attempt++) {
    const res = await fetch('/api/chat', {
      method: 'POST',
      body: JSON.stringify(body),
    });
    if (res.status !== 429) return res;
    const ra = res.headers.get('Retry-After');
    const wait = Number(ra) * 1000 || 500 * 2 ** attempt;
    await sleep(wait + Math.random() * 250);
  }
  throw new Error('Rate limited');
}