Useful Cheatsheets
Build / AI /

Audio and Speech

Transcribe spoken audio, generate timestamped subtitles, and synthesize natural text-to-speech with Whisper and OpenAI.

TL;DR

  1. Transcribe spoken audio recordings into text using Whisper transcriptions.create() calls.
  2. Request word-level timestamps using verbose_json response formatting options.
  3. Synthesize natural speech audio streams utilizing OpenAI audio.speech.create() APIs.
Jump to section

Whisper Transcription API

    Basic Audio File Transcription

    Transcribe local audio recording to plain text string.

    import OpenAI from 'openai';
import fs from 'node:fs';
const client = new OpenAI();

const tr = await client.audio.transcriptions.create({
  file: fs.createReadStream('meeting.mp3'),
  model: 'whisper-1',
});
console.log(tr.text);
    Domain Glossary Prompting

    Guide Whisper spelling for technical jargon and proper nouns.

    const tr = await client.audio.transcriptions.create({
  file: fs.createReadStream('podcast.m4a'),
  model: 'whisper-1',
  prompt: 'Cheatsheet, TypeScript, PostgreSQL, Zod.',
});
    Automatic Audio Translation

    Transcribe foreign language speech directly into English.

    const tr = await
  client.audio.translations.create({
    file: fs.createReadStream('spanish_call.mp3'),
    model: 'whisper-1',
  });
console.log(tr.text);

Timestamps & Subtitle Generation

    Word-Level Timestamp Request

    Request granular millisecond boundaries for video subtitles.

    const out = await client.audio.transcriptions.create({
  file: fs.createReadStream('clip.mp3'),
  model: 'whisper-1',
  response_format: 'verbose_json',
  timestamp_granularities: ['word', 'segment'],
});
const words = (out as any).words;
    SRT Subtitle File Generator

    Format segment timestamps into standard SubRip format.

    function toSRTTime(seconds: number): string {
  const ms = Math.floor((seconds % 1) * 1000);
  const s = Math.floor(seconds % 60);
  const m = Math.floor((seconds / 60) % 60);
  const h = Math.floor(seconds / 3600);
  const p = (n: number, d = 2) => {
    return String(n).padStart(d, '0');
  };
  return `${p(h)}:${p(m)}:${p(s)},${p(ms, 3)}`;
}
    Segment Iterator

    Iterate across subtitle blocks and construct SRT cues.

    function buildSRT(segments: any[]) {
  return segments.map((seg, i) => {
    const start = toSRTTime(seg.start);
    const end = toSRTTime(seg.end);
    const time = `${start} --> ${end}`;
    const body = `${time}\n${seg.text.trim()}`;
    return `${i + 1}\n${body}\n`;
  }).join('\n');
}

Text-to-Speech (TTS) Synthesis

    Lifelike Voice Audio Generation

    Synthesize spoken narration from text script.

    const mp3 = await client.audio.speech.create({
  model: 'tts-1',
  voice: 'alloy', // alloy, echo, fable, onyx, nova
  input: 'Welcome to Useful Cheatsheets.',
});
const buffer = Buffer.from(await mp3.arrayBuffer());
await fs.promises.writeFile('welcome.mp3', buffer);
    HD Model Voice Selection

    Use tts-1-hd for master production audio tracks.

    const hd = await client.audio.speech.create({
  model: 'tts-1-hd',
  voice: 'nova',
  input: 'High-definition voice mastering.',
  response_format: 'mp3',
  speed: 1.05,
});
    Low-Latency Opus Streaming

    Stream spoken audio chunks with Opus codec for real-time apps.

    const stream = await client.audio.speech.create({
  model: 'tts-1',
  voice: 'echo',
  input: 'Real-time assistant response.',
  response_format: 'opus',
});
// Pipe directly to client web socket

Audio Preprocessing & File Limits

    File Size Gate Check

    Verify audio file adheres to 25MB HTTP upload ceiling.

    function checkAudioSize(filePath: string) {
  const stat = fs.statSync(filePath);
  const mb = stat.size / (1024 * 1024);
  if (mb > 24) {
    const s = mb.toFixed(1);
    throw new Error(`File ${s}MB exceeds 25MB`);
  }
}
    Segment Audio Splitter Concept

    Slice long recordings into 10-minute segments before upload.

    // ffmpeg -i in.mp3 -f segment -segment_time 600
// -c copy part%03d.mp3
// Process parts through Whisper and merge transcripts
    Transcript Merging Sanitizer

    Join sliced audio transcript segments into unified text.

    function mergeTranscripts(parts: string[]) {
  return parts
    .map(p => p.trim())
    .filter(Boolean)
    .join(' ');
}

Tips

  1. Specify custom glossaries and acronyms inside the prompt parameter of Whisper to dramatically improve transcription accuracy.
  2. Chunk audio files larger than 25 megabytes into smaller segments with ffmpeg before transmitting to provider endpoints.

Warnings

  1. Never send raw uncompressed WAV audio across network boundaries when lightweight mp3 or m4a files preserve identical acoustic fidelity.
  2. Avoid blocking main event loops during long speech generation requests by utilizing streaming toReadableStream() pipe mechanics.

In Practice

FAQ