Chain-of-Thought Mechanics

Chain-of-thought (CoT) prompting is the discovery that LLMs are often wildly better at multi-step problems when asked to reason aloud rather than answer in one shot. Wei et al. (2022) documented this effect across math, logic, and commonsense reasoning.

How CoT works mechanically

1. 1Each token the model writes becomes part of its future context
2. 2Intermediate steps give the model a 'scratchpad' for calculation
3. 3Complex inferences can be decomposed into simple per-token steps
4. 4The final answer is conditioned on the scratchpad, reducing shortcut errors

Without CoT: Q: If I have 5 apples and buy 3 more, then eat 2, how many do I have? A: 6 With CoT: Q: (same) A: Let me think step by step. Start with 5 apples. Buy 3 more → 5 + 3 = 8. Eat 2 → 8 - 2 = 6. The answer is 6.

Reasoning models take this further

Since 2024, models like OpenAI's o1, o3, Claude with extended thinking, and DeepSeek-R1 have been explicitly trained to reason for a long time before answering. Test-time compute is now a core axis of capability, distinct from parameter scaling.

When CoT goes wrong

• Overconfidence: verbose reasoning does not prevent wrong answers — it can make them sound more certain
• Hallucinated steps: the model invents intermediate calculations
• Unfaithful reasoning: the stated steps do not reflect the actual decision
• Wasted tokens: simple questions do not benefit and cost more

“Chain-of-thought prompting significantly improves the ability of large language models to perform complex reasoning.”

The big idea: making thinking visible usually makes thinking better. But visible is not the same as faithful — and that distinction matters.