Simpson's Paradox: When Aggregated Data Lies

A trend that appears in every subgroup can reverse when you combine the groups. This is Simpson's Paradox, and it hides in plain sight.

30 min · Reviewed 2026

A Famous Medical Case

Imagine a study of two treatments for kidney stones. Treatment A beats Treatment B for small stones. Treatment A also beats Treatment B for large stones. But when you combine all patients, Treatment B looks better overall. This actually happened in real medical data. It is Simpson's Paradox.

A toy example

Subgroup	Treatment A	Treatment B
Small stones	93% cured (81/87)	87% cured (234/270)
Large stones	73% cured (192/263)	69% cured (55/80)
Overall	78% cured (273/350)	83% cured (289/350)

Where Simpson's Paradox appears

Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in almost every department (women applied to more competitive departments)
COVID-19 case fatality: overall rates can flip when you stratify by age
A/B test results where a minority group reverses the majority trend
School rankings: combined scores can mislead when student populations differ

The confounder concept

Simpson's Paradox happens when there is a confounding variable, an unmeasured factor that affects both the input and the outcome. In the kidney stone case, stone size is the confounder. It affects both treatment choice (doctors pick A for harder cases) and cure rate (large stones are harder to treat).

import pandas as pd df = pd.read_csv('treatment_data.csv') # Overall rates (deceptive) print(df.groupby('treatment')['cured'].mean()) # Disaggregated by severity (honest) print(df.groupby(['severity', 'treatment'])['cured'].mean()) # Same analysis as a pivot table print(pd.pivot_table(df, index='treatment', columns='severity', values='cured', aggfunc='mean', margins=True))Always check disaggregated rates

The big idea: the total is not always the truth. Always slice your data by relevant subgroups before drawing conclusions. Aggregation can reverse reality.

End-of-lesson check

8 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-data-simpsons-paradox

What is the main idea of "Simpson's Paradox: When Aggregated Data Lies"?
1. A trend that appears in every subgroup can reverse when you combine the groups. This is Simpson's Paradox, and it hides in plain sight.
2. Use AI as the final authority for the whole decision
3. Avoid checking the answer once it sounds polished
4. Focus only on speed instead of judgment
Which concept is most central to "Simpson's Paradox: When Aggregated Data Lies"?
1. aggregation
2. Simpson's paradox
3. confounders
4. confounder
Which use of AI fits this topic best?
1. Let the AI decide what matters without your review
2. Use the answer before checking whether it fits the situation
3. Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in almost every department.
4. Treat the AI output as automatically correct
What should a careful learner remember about "How can this happen?"?
1. Use AI to draft or organize ideas about Simpson's paradox, then verify before acting.
2. Skip the context so the tool can guess faster
3. Treat the output as private even after sharing it online
4. Use the answer without checking the source
You want to use AI after this lesson. What is the safest next step?
1. Act immediately because the AI answer is written clearly
2. Use AI for drafting and comparison, but verify before publishing or relying on it.
3. Hide uncertainty so the final answer looks cleaner
4. Use private or sensitive details before checking permission
How should AI output about Simpson's paradox be treated?
1. As proof that no other source is needed
2. As a replacement for context, consent, or expert review
3. As a draft or helper output that still needs human judgment and verification
4. As something that becomes correct when it sounds confident
Name one way to verify an AI answer about Simpson's paradox.
Which action would help you apply "Simpson's Paradox: When Aggregated Data Lies" responsibly?
1. Use the tool to avoid thinking through the tradeoff
2. Keep going even if the output conflicts with a trusted source
3. Treat the AI output as automatically correct
4. COVID-19 case fatality: overall rates can flip when you stratify by age

← Back to interactive lesson

Tendril · Creators · AI Foundations