Streaming AI Responses
Render AI output token by token as it streams in, so users see progress instead of a spinner.
TL;DR
- Read the response body with a
ReadableStreamreader. - Decode chunks and append tokens to
stateas they arrive. - Show partial text immediately instead of a
spinner.
Read the Stream
Call the EndpointPOST the prompt to your own streaming route.
const res = await fetch('/api/chat', {
method: 'POST',
body: JSON.stringify({ prompt }),
});Get a ReaderRead the response body as a byte stream.
const reader = res.body.getReader();
const decoder = new TextDecoder();Loop the ChunksRead until done, decoding each chunk to text.
while (true) {
const { value, done } = await reader.read();
if (done) break;
handle(decoder.decode(value));
}Parse SSE Events
Split on LinesEach event is a line that starts with data:.
for (const line of chunk.split('\n')) {
if (!line.startsWith('data: ')) continue;
const json = line.slice(6);
}Parse the TokenPull the text delta out of each JSON event.
const evt = JSON.parse(json);
const token = evt.delta ?? '';Stop at [DONE]A sentinel line marks the end of the stream.
if (json.trim() === '[DONE]') return;Append to State
Append TokensAdd each token to the growing message text.
setText((prev) => prev + token);Buffer in a RefCollect tokens to avoid a render per token.
bufferRef.current += token;Flush on a FramePush the buffer to state once per paint.
requestAnimationFrame(() =>
setText(bufferRef.current));Clean Up
Pass a SignalGive fetch a signal so a new send can cancel.
fetch(url, { signal: controller.signal });Release the ReaderCancel the reader if the user leaves.
reader.cancel();Catch AbortsIgnore AbortError; surface real failures.
catch (e) {
if (e.name !== 'AbortError') setError(e);
}Tips
- Pass fetch an abort
signaland read the reader in a loop, updating state on each chunk so React re-renders the growing text. - If tokens arrive faster than the browser paints, buffer them in a
refand flush to state once per animation frame.
Warnings
- Never call the AI provider directly from the browser; stream through your own
routeHandlerso keys stay on the server. - Always handle a dropped connection and release the reader, or a half-finished
streamcan leave the UI stuck mid-answer.
In Practice
A minimal hook that streams an AI answer into React state, token by token.
- fetch calls your server route and returns a streaming body reader.
- The loop decodes each chunk and appends the text to state.
- setText on each chunk re-renders the partial answer immediately.
- Returning text and send lets any component show live output.
function useStream() {
const [text, setText] = useState('');
const send = async (prompt: string) => {
setText('');
const res = await fetch('/api/chat', {
method: 'POST',
body: JSON.stringify({ prompt }),
});
const reader = res.body!.getReader();
const decoder = new TextDecoder();
while (true) {
const { value, done } = await reader.read();
if (done) break;
setText((t) => t + decoder.decode(value));
}
};
return { text, send };
}FAQ
Fetch a streaming endpoint, then read response.body.getReader() in a loop. Decode each chunk with a TextDecoder, append the text to state, and React re-renders the partial answer. Users see words appear instead of waiting for the whole response.
Most use server-sent events: lines like data: {...}. Split incoming chunks on newlines, parse each data: line as JSON, and pull out the token. A [DONE] sentinel line marks the end of the stream.
You are probably setting state on every tiny chunk. Append to a ref and flush to state on an animation frame, or buffer a few chunks. This cuts re-renders while keeping the text visibly flowing.