Why does chain-of-thought prompting improve reasoning accuracy?

Autoregressive transformers compute the next token based strictly on preceding tokens. By forcing the model to generate intermediate reasoning steps, each subsequent calculation can attend to previous deduction steps rather than guessing immediately.

What is self-consistency in chain-of-thought?

Self-consistency is an ensemble technique where you prompt a model multiple times at temperature 0.7 to generate diverse reasoning paths. You then take a majority vote across the final answers to select the most reliable result.

How do thinking blocks work in Claude 3.7?

Claude supports extended thinking with explicit blocks or native reasoning modes. The model outputs its analytical scratchpad before delivering its final answer, giving developers insight into deduction paths.

Build / AI /

Chain of Thought

Elicit step-by-step reasoning, mathematical problem solving, and complex logic through explicit chain-of-thought prompting.

TL;DR

Instruct complex models to think step-by-step using chain-of-thought prompts.
Isolate intermediate calculation scratchpads inside explicit <thinking> XML blocks.
Sample multiple reasoning paths and select consensus via self-consistency.

Jump to section

Reasoning Elicitation Triggers
Scratchpad Parsing In TypeScript
Self-Consistency Sampling
Frontier Reasoning Integration
Tips
Warnings
In Practice
FAQ

Reasoning Elicitation Triggers

Zero-Shot CoT Trigger

Classic heuristic phrase activating step-by-step inference.

Think step-by-step before answering.

Structured Scratchpad Prompt

Direct intermediate logic into dedicated reasoning enclosure.

First, outline steps in <thinking>...</thinking> tags.
Then provide final answer in <answer>.

Algorithmic Verification Step

Instruct model to audit its own calculation before concluding.

Verify math steps for errors
before emitting final sum.

Scratchpad Parsing In TypeScript

Extract Thinking Block

Isolate reasoning tokens from user-facing answer via regex.

function extractCoT(raw: string) {
  const p = /<thinking>([\s\S]*?)<\/thinking>/;
  const think = raw.match(p);
  const ans = raw.replace(p, '');
  return {
    reasoning: think?.[1]?.trim(),
    answer: ans.trim(),
  };
}

Stream Parsing Scratchpad

Differentiate live reasoning tokens from final markdown output.

let inThinking = false;
if (chunk.includes('<thinking>')) inThinking = true;
if (chunk.includes('</thinking>')) inThinking = false;

Collapsible UI Renderer

Render thinking process in an expandable disclosure component.

// Collapsible UI pattern
// Render thought disclosure header
// Display reasoning scratchpad
// Render final answer

Self-Consistency Sampling

Parallel Sample Generation

Dispatch multiple parallel completion calls at higher temperature.

const runs = [1, 2, 3, 4, 5];
const samples = await Promise.all(runs.map(() =>
  client.chat.completions.create({
    model: 'gpt-4o',
    messages: [{ role: 'user', content: mathPrompt }],
    temperature: 0.7,
  })
));

Majority Voting Algorithm

Select final answer supported by consensus of reasoning paths.

function majorityVote(answers: string[]): string {
  const counts: Record<string, number> = {};
  for (const a of answers) {
    counts[a] = (counts[a] || 0) + 1;
  }
  return Object.keys(counts).reduce((a, b) =>
    counts[a] > counts[b] ? a : b
  );
}

Outlier Rejection

Filter out reasoning paths that failed basic consistency checks.

const valid = samples.filter(s => isValidAnswer(s));
const winner = majorityVote(valid);

Frontier Reasoning Integration

OpenAI Reasoning Effort

Tune reasoning token budget on models like o3-mini.

const res = await openai.chat.completions.create({
  model: 'o3-mini',
  reasoning_effort: 'medium', // low | medium | high
  messages: [{ role: 'user', content: logicPuzzle }],
});

Anthropic Extended Thinking

Allocate explicit thinking budget in tokens for Claude.

const res = await anthropic.messages.create({
  model: 'claude-3-7-sonnet-20250219',
  max_tokens: 4000,
  thinking: { type: 'enabled', budget_tokens: 2048 },
  messages: [{ role: 'user', content: puzzle }],
});

Reasoning Token Metrics

Monitor billable hidden thinking tokens in response usage.

const reasoningTokens =
  (res.usage as any)?.completion_tokens_details
    ?.reasoning_tokens ?? 0;
console.log(`Reasoning tokens: ${reasoningTokens}`);

Tips

Direct the model to articulate its complete logical deduction into explicit intermediate-tokens before emitting final answers.
Parse out and hide internal <thinking> scratchpad tags before displaying final responses in customer-facing user interfaces.

Warnings

Do not demand immediate answers without allowing intermediate reasoning-tokens because autoregressive models cannot perform multi-step math instantly.
Be aware that deep reasoning models like o3-mini generate internal hidden reasoning tokens that bill as standard output tokens.

In Practice

Executes a multi-step math problem using an explicit thinking scratchpad and parses out the final answer.

Instruct model to output its intermediate deduction inside thinking tags.
Dispatch request to model with reasoning problem.
Parse thinking scratchpad and clean final answer.
Log intermediate reasoning and final conclusion.

import OpenAI from 'openai';

const client = new OpenAI();

async function solveMath(problem: string) {
  const sys = 'Write <thinking> then <answer>.';
  const res = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [
      { role: 'system', content: sys },
      { role: 'user', content: problem },
    ],
  });
  const txt = res.choices[0]?.message?.content ?? '';
  const m = txt.match(/<answer>([\s\S]*?)<\/answer>/);
  return m ? m[1].trim() : txt;
}

const ans = await solveMath('3 pens cost $6. 9 pens?');
console.log(ans);