Lesson 631 of 1596
Few-Shot Example Curation: Quality, Rotation, and Counter-Examples, Part 1
Chain-of-thought prompts show real performance gains on reasoning tasks — and zero benefit on tasks that don't need reasoning. Here's how to tell which is which.
Creators · Prompting · ~24 min read
The premise
Chain-of-thought is not a universal upgrade; it helps on reasoning-bound tasks and is overhead everywhere else.
What AI does well here
- Use CoT on tasks requiring multi-step reasoning (math, complex logic, multi-constraint problems)
- Use few-shot CoT examples on reasoning tasks where the structure of reasoning matters
- Hide CoT from end-user output when the reasoning isn't user-facing value
- Evaluate with and without CoT to confirm benefit on YOUR task
What AI cannot do
- Make non-reasoning tasks better with CoT (it just adds tokens)
- Make CoT a substitute for fine-tuning on hard reasoning tasks
- Trust the reasoning trace as ground truth (models can produce plausible-but-wrong reasoning)
Key terms in this lesson
End-of-lesson quiz
Check what stuck
10 questions · Score saves to your progress.
Tutor
Curious about “Few-Shot Example Curation: Quality, Rotation, and Counter-Examples, Part 1”?
Ask anything about this lesson. I’ll answer using just what you’re reading — short, friendly, grounded.
Progress saved locally in this browser. Sign in to sync across devices.
Related lessons
Keep going
Creators · 40 min
Chain-of-Thought for Production: When It Helps, When It Hurts, Part 2
Use a reasoning step that you discard before showing the final answer.
Builders · 40 min
Chain-of-Thought for Builders: Make AI Show Its Reasoning
Force AI to explain its reasoning out loud, and you'll catch its mistakes faster.
Builders · 24 min
Chain-of-Thought: Make the AI Show Its Work
Telling the AI to 'think step by step' before answering dramatically improves its accuracy on reasoning problems. Here's why and when.