Useful Cheatsheets
Build / AI /

AI Streaming Responses

Stream AI completions in real time using Server-Sent Events, async chunk iterators, and readable web streams.

TL;DR

  1. Enable streaming mode by passing the stream: true request option.
  2. Consume incoming text tokens progressively using for await...of loops.
  3. Transmit chunks to browser clients using standard Server-Sent-Events headers.
Jump to section

SDK Streaming Basics

    OpenAI Stream Flag

    Request streaming response from OpenAI using async chunk iterator.

    const stream = await openai.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Count 1-10' }],
  stream: true,
});
    Async Iterator Loop

    Iterate over incoming stream chunks progressively as tokens arrive.

    for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content ?? '';
  process.stdout.write(delta);
}
    Claude Stream Helper

    Use Anthropic stream event helper for streamlined text handling.

    const stream = client.messages.stream({
  model: 'claude-3-5-sonnet-20241022',
  max_tokens: 500,
  messages: [{ role: 'user', content: 'Hi' }],
}).on('text', (d) => process.stdout.write(d));

Server-Sent Events (SSE)

    SSE Response Headers

    Configure HTTP response headers for real-time text event streaming.

    res.writeHead(200, {
  'Content-Type': 'text/event-stream',
  'Cache-Control': 'no-cache',
  'Connection': 'keep-alive',
});
    Event Serialization

    Format chunk payloads according to standard EventSource spec.

    function writeSSE(res: Response, text: string) {
  res.write(`data: ${JSON.stringify({ text })}\n\n`);
}
    Stream End Sentinel

    Emit termination signal so client closes EventSource connection.

    res.write('data: [DONE]\n\n');
res.end();

Web Streams And Next.js

    ReadableStream Pipeline

    Construct modern Web API ReadableStream for edge runtimes.

    const stream = new ReadableStream({
  async start(controller) {
    for await (const chunk of aiStream) {
      controller.enqueue(encoder.encode(chunk));
    }
    controller.close();
  }
});
    AI SDK streamText

    Leverage Vercel AI SDK to stream text directly to client components.

    import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

const r = streamText({
  model: openai('gpt-4o'),
  prompt,
});
return r.toDataStreamResponse();
    Client Disconnect Abort

    Halt upstream AI generation immediately when HTTP socket closes.

    req.on('close', () => {
  abortController.abort();
  console.log('Client aborted stream');
});

Stream Buffering And State

    Full Text Accumulator

    Buffer individual deltas into complete string for persistence.

    let fullResponse = '';
for await (const chunk of stream) {
  const token = chunk.choices[0]?.delta?.content ?? '';
  fullResponse += token;
}
    Throttled UI Updates

    Batch rapid token arrival to prevent front-end render lag.

    let pending = '';
setInterval(() => {
  if (pending) { updateUI(pending); pending = ''; }
}, 50);
    Final Usage Extraction

    Extract final usage statistics from final stream summary chunk.

    const final = await stream.finalMessage();
const inTok = final.usage.input_tokens;
console.log(`Total tokens used: ${inTok}`);

Tips

  1. Render incoming tokens directly into user interface elements to reduce perceived wait times from seconds to under 400 milliseconds via ReadableStream.
  2. Accumulate streamed chunks into a server-side fullResponse buffer to persist the final combined completion message in your database.

Warnings

  1. Do not parse incomplete JSON tokens during streaming; wait for the complete stream to finish before invoking JSON.parse.
  2. Ensure server connections are cleaned up when clients disconnect by listening for browser req.on('close') abort events.

In Practice

FAQ