Useful Cheatsheets
Build / AI / AI Development

Prompt Caching

Slash AI latency and token costs by up to 90 percent using Anthropic and OpenAI prompt caching breakpoints.

TL;DR

  1. Put stable system content first and changing user content last.
  2. Mark Claude cache breakpoints with cache_control; OpenAI caches prefixes automatically.
  3. Confirm hits by checking cache_read_input_tokens in the usage data.
Jump to section

Anthropic Cache Breakpoints

    System Cache Control

    Attach ephemeral cache breakpoint to static system instructions.

    const system = [
  {
    type: 'text',
    text: largeStaticDocumentation,
    cache_control: { type: 'ephemeral' },
  },
];
    Tool Cache Breakpoints

    Cache extensive tool definition schemas to avoid re-billing.

    const cachedTools = tools.map((t, i) => ({
  ...t,
  ...(i === tools.length - 1
    ? { cache_control: { type: 'ephemeral' } }
    : {}),
}));
    Multiple Breakpoint Limits

    Configure up to four distinct cache checkpoints per request.

    // Breakpoint 1: System prompt
// Breakpoint 2: Static reference book
// Breakpoint 3: Conversation history
// Dynamic: Final user question

OpenAI Automatic Caching

    Prefix Alignment

    Order prompt components strictly from static to dynamic.

    // 1. Static system prompt (> 1024 tokens)
// 2. Static documents
// 3. Dynamic runtime user query
// OpenAI caches shared prefixes automatically
    Inspect Cached Tokens

    Verify cache hits through prompt_tokens_details usage field.

    const u = res.usage as any;
const cached =
  u.prompt_tokens_details?.cached_tokens ?? 0;
console.log(`Cached: ${cached}`);
    Exact Prefix Match

    Ensure identical character prefix to achieve cache hits.

    const match = prev.startsWith(prefix);
// Any change invalidates subsequent cache

Cache Hit Verification

    Anthropic Cache Usage

    Read cache creation and cache read metrics from response.

    const u = res.usage;
const w = u.cache_creation_input_tokens;
const r = u.cache_read_input_tokens;
console.log(`Write: ${w}, Read: ${r}`);
    Calculate Cost Savings

    Compute monetary savings realized through cached tokens.

    function calcSavings(
  readTokens: number,
  rate: number
) {
  const full = (readTokens / 1e6) * rate;
  return full * 0.90; // 90% cache discount
}
    Cache TTL Monitoring

    Track 5-minute ephemeral cache window expiration.

    const isWarm = Date.now() - lastRequestTime < 300000;
// Keep cache warm with recurring queries

Architectural Best Practices

    Document Pinning

    Pin massive background context above conversation turns.

    const messages = [
  { role: 'user', content: cachedDocBlock },
  { role: 'user', content: dynamicUserQuestion },
];
    Remove Dynamic Variables

    Strip dynamic timestamps and request IDs from cached blocks.

    // Bad: `Current time: ${new Date().toISOString()}`
// Good: Place timestamp in dynamic user prompt only
    Multi-Turn Dialogue Caching

    Cache conversation history up to the second-to-last turn.

    const turns = history.map((h, i) => ({
  ...h,
  ...(i === history.length - 2
    ? { cache_control: { type: 'ephemeral' } }
    : {}),
}));

Tips

  1. Put static context first and the changing user question last, so cache_read_input_tokens stays high on every request.
  2. The default cache lasts five minutes and refreshes on every hit. For slower traffic, set ttl: '1h' on the breakpoint instead of sending keep-alive pings.

Warnings

  1. Avoid timestamps or random IDs in the system prompt, because changing even one token invalidates everything cached after it.
  2. Check the minimum cacheable prefix for your model, roughly 512 to 4096 tokens, because shorter prompts silently skip cache_control.

In Practice

FAQ