Simpson's Paradox: When Aggregated Data Lies

A trend that appears in every subgroup can reverse when you combine the groups. This is Simpson's Paradox, and it hides in plain sight.

30 min · Reviewed 2026

A Famous Medical Case

Imagine a study of two treatments for kidney stones. Treatment A beats Treatment B for small stones. Treatment A also beats Treatment B for large stones. But when you combine all patients, Treatment B looks better overall. This actually happened in real medical data. It is Simpson's Paradox.

A toy example

Subgroup	Treatment A	Treatment B
Small stones	93% cured (81/87)	87% cured (234/270)
Large stones	73% cured (192/263)	69% cured (55/80)
Overall	78% cured (273/350)	83% cured (289/350)

Where Simpson's Paradox appears

Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in almost every department (women applied to more competitive departments)
COVID-19 case fatality: overall rates can flip when you stratify by age
A/B test results where a minority group reverses the majority trend
School rankings: combined scores can mislead when student populations differ

The confounder concept

Simpson's Paradox happens when there is a confounding variable, an unmeasured factor that affects both the input and the outcome. In the kidney stone case, stone size is the confounder. It affects both treatment choice (doctors pick A for harder cases) and cure rate (large stones are harder to treat).

import pandas as pd

df = pd.read_csv('treatment_data.csv')

# Overall rates (deceptive)
print(df.groupby('treatment')['cured'].mean())

# Disaggregated by severity (honest)
print(df.groupby(['severity', 'treatment'])['cured'].mean())

# Same analysis as a pivot table
print(pd.pivot_table(df, 
    index='treatment', 
    columns='severity', 
    values='cured', 
    aggfunc='mean',
    margins=True))Always check disaggregated rates

The big idea: the total is not always the truth. Always slice your data by relevant subgroups before drawing conclusions. Aggregation can reverse reality.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-data-simpsons-paradox

What is the core idea behind "Simpson's Paradox: When Aggregated Data Lies"?
1. A trend that appears in every subgroup can reverse when you combine the groups. This is Simpson's Paradox, and it hides in plain sight.
2. Probe for edge cases: long-tail inputs, adversarial tests
3. dataflow
4. ai.txt
Which term best describes a foundational idea in "Simpson's Paradox: When Aggregated Data Lies"?
1. confounder
2. Simpson's paradox
3. stratification
4. aggregation
A learner studying Simpson's Paradox: When Aggregated Data Lies would need to understand which concept?
1. Simpson's paradox
2. stratification
3. confounder
4. aggregation
Which of these is directly relevant to Simpson's Paradox: When Aggregated Data Lies?
1. Simpson's paradox
2. confounder
3. aggregation
4. stratification
Which of the following is a key point about Simpson's Paradox: When Aggregated Data Lies?
1. Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in alm…
2. COVID-19 case fatality: overall rates can flip when you stratify by age
3. A/B test results where a minority group reverses the majority trend
4. School rankings: combined scores can mislead when student populations differ
Which of these does NOT belong in a discussion of Simpson's Paradox: When Aggregated Data Lies?
1. A/B test results where a minority group reverses the majority trend
2. COVID-19 case fatality: overall rates can flip when you stratify by age
3. Probe for edge cases: long-tail inputs, adversarial tests
4. Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in alm…
What is the key insight about "How can this happen?" in the context of Simpson's Paradox: When Aggregated Data Lies?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. dataflow
3. Treatment A was preferred for hard cases (large stones). Treatment B was given more often for easy cases (small stones).
4. ai.txt
What is the key insight about "The cure: stratify" in the context of Simpson's Paradox: When Aggregated Data Lies?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. dataflow
3. ai.txt
4. When groups have different baselines, never trust the aggregate without checking each subgroup.
Which statement accurately describes an aspect of Simpson's Paradox: When Aggregated Data Lies?
1. Imagine a study of two treatments for kidney stones. Treatment A beats Treatment B for small stones.
2. Probe for edge cases: long-tail inputs, adversarial tests
3. dataflow
4. ai.txt
What does working with Simpson's Paradox: When Aggregated Data Lies typically involve?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. Simpson's Paradox happens when there is a confounding variable, an unmeasured factor that affects both the input and the outcome.
3. dataflow
4. ai.txt
Which of the following is true about Simpson's Paradox: When Aggregated Data Lies?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. dataflow
3. The big idea: the total is not always the truth. Always slice your data by relevant subgroups before drawing conclusions.
4. ai.txt
Which best describes the scope of "Simpson's Paradox: When Aggregated Data Lies"?
1. It is unrelated to foundations workflows
2. It applies only to the opposite beginner tier
3. It was deprecated in 2024 and no longer relevant
4. It focuses on A trend that appears in every subgroup can reverse when you combine the groups. This is Simpson's Pa
Which section heading best belongs in a lesson about Simpson's Paradox: When Aggregated Data Lies?
1. A toy example
2. Probe for edge cases: long-tail inputs, adversarial tests
3. dataflow
4. ai.txt
Which section heading best belongs in a lesson about Simpson's Paradox: When Aggregated Data Lies?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. Where Simpson's Paradox appears
3. dataflow
4. ai.txt
Which section heading best belongs in a lesson about Simpson's Paradox: When Aggregated Data Lies?
1. Probe for edge cases: long-tail inputs, adversarial tests
2. dataflow
3. The confounder concept
4. ai.txt

← Back to interactive lesson

Tendril · Creators · AI Foundations