Lesson 7 of 1570
Is the Model Reasoning or Pattern Matching?
The line between deep reasoning and clever pattern recognition is blurry. Here's how researchers try to tell them apart.
Lesson map
What this lesson covers
Learning path
The main moves in order
- 1The Big Debate
- 2reasoning
- 3chain of thought
- 4memorization
Concept cluster
Terms to connect while reading
Section 1
The Big Debate
When an AI solves a math problem, is it actually reasoning, or is it matching patterns from its training set? Smart researchers disagree, and the answer probably depends on the problem.
Evidence for pattern matching
- Performance drops sharply on problems slightly out of distribution
- Changing variable names in a math problem can flip the answer
- Models often cannot solve problems they were not exposed to in training
Evidence for real reasoning
- Chain-of-thought prompting improves performance on novel problems
- Models can combine steps across topics they were not jointly trained on
- Tool use and multi-step planning beat pure recall
Reasoning models are a category now
Models like o1 and Claude with extended thinking explicitly spend more compute on harder problems. They think longer, try alternatives, and backtrack. These are not just bigger chatbots — they are trained specifically to reason.
Compare the options
| Standard LLM | Reasoning model |
|---|---|
| Fixed compute per token | Variable compute per problem |
| One-pass generation | Internal think-try-revise loops |
| Fast but shallow on hard problems | Slower but better on hard problems |
| Cheaper per call | More expensive per call |
How to probe your own model
- 1Change surface details but keep the logic identical. Does performance hold?
- 2Ask the model to show its work. Does the work match the answer?
- 3Present a problem that requires combining two fields. Can it actually combine?
- 4Give an impossible problem. Does it say so, or does it fabricate?
“Intelligence is the ability to solve problems you have not seen before.”
Key terms in this lesson
The big idea: modern models blend memorization and reasoning. Good prompts and good evals are the only way to know which one you are getting for any given task.
End-of-lesson quiz
Check what stuck
15 questions · Score saves to your progress.
Tutor
Curious about “Is the Model Reasoning or Pattern Matching?”?
Ask anything about this lesson. I’ll answer using just what you’re reading — short, friendly, grounded.
Progress saved locally in this browser. Sign in to sync across devices.
Related lessons
Keep going
Builders · 28 min
BLEU, ROUGE, F1 — Automatic Metrics and Their Limits
Before LLMs-as-judges, researchers had hand-made metrics. They still matter — and still mislead.
Builders · 28 min
Bayesian Reasoning for Everyday Life
Bayes' rule is just 'update your belief with evidence.' It is shockingly useful.
Builders · 22 min
What a Spreadsheet Actually Is
Excel and Google Sheets hide a lot of complexity behind a pretty grid. Once you see what is really happening, you will never look at a spreadsheet the same way.