Simpson's Paradox: When Aggregated Data Lies

Imagine a study of two treatments for kidney stones. Treatment A beats Treatment B for small stones. Treatment A also beats Treatment B for large stones. But when you combine all patients, Treatment B looks better overall. This actually happened in real medical data. It is Simpson's Paradox.

A toy example

Where Simpson's Paradox appears

• Berkeley admissions 1973: overall lower acceptance rate for women, but women had higher rates in almost every department (women applied to more competitive departments)
• COVID-19 case fatality: overall rates can flip when you stratify by age
• A/B test results where a minority group reverses the majority trend
• School rankings: combined scores can mislead when student populations differ

The confounder concept

Simpson's Paradox happens when there is a confounding variable, an unmeasured factor that affects both the input and the outcome. In the kidney stone case, stone size is the confounder. It affects both treatment choice (doctors pick A for harder cases) and cure rate (large stones are harder to treat).

import pandas as pd df = pd.read_csv('treatment_data.csv') # Overall rates (deceptive) print(df.groupby('treatment')['cured'].mean()) # Disaggregated by severity (honest) print(df.groupby(['severity', 'treatment'])['cured'].mean()) # Same analysis as a pivot table print(pd.pivot_table(df, index='treatment', columns='severity', values='cured', aggfunc='mean', margins=True))

The big idea: the total is not always the truth. Always slice your data by relevant subgroups before drawing conclusions. Aggregation can reverse reality.