Simpson's Paradox: When Aggregated Data Lies

Imagine a study of two treatments for kidney stones. Treatment A beats Treatment B for small stones. Treatment A also beats Treatment B for large stones. But when you combine all patients, Treatment B looks better overall. This actually happened in real medical data. It is Simpson's Paradox.

A toy example

Where Simpson's Paradox appears

• Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in almost every department (women applied to more competitive departments)
• COVID-19 case fatality: overall rates can flip when you stratify by age
• A/B test results where a minority group reverses the majority trend
• School rankings: combined scores can mislead when student populations differ

The confounder concept

Simpson's Paradox happens when there is a confounding variable, an unmeasured factor that affects both the input and the outcome. In the kidney stone case, stone size is the confounder. It affects both treatment choice (doctors pick A for harder cases) and cure rate (large stones are harder to treat).

import pandas as pd

df = pd.read_csv('treatment_data.csv')

# Overall rates (deceptive)
print(df.groupby('treatment')['cured'].mean())

# Disaggregated by severity (honest)
print(df.groupby(['severity', 'treatment'])['cured'].mean())

# Same analysis as a pivot table
print(pd.pivot_table(df, 
    index='treatment', 
    columns='severity', 
    values='cured', 
    aggfunc='mean',
    margins=True))

The big idea: the total is not always the truth. Always slice your data by relevant subgroups before drawing conclusions. Aggregation can reverse reality.