Measurement Bias: When the Ruler Is Bent

You want to predict which students will succeed in college. You don't have a success score, so you use GPA. But GPA depends on which teachers a student had, their sleep schedule, whether their family had tutors, and a hundred other things. GPA is a proxy for success, and every proxy has errors baked in.

Famous examples

The healthcare algorithm scandal

A 2019 study in Science by Obermeyer et al. examined a widely-used US healthcare algorithm that decided which patients needed extra care. It used past spending to predict future need. Because Black patients historically received less care, they also spent less, so the algorithm systematically underestimated their needs. The algorithm was assigning white patients with mild illness to the same care tier as Black patients with severe illness.

How to spot measurement bias

1. 1Ask: is the measurement the thing I care about, or a proxy?
2. 2Ask: could two groups with the same true value get different measured values?
3. 3Look for historical artifacts embedded in the measurement process
4. 4Cross-check predictions against an independent direct measurement
5. 5Consult affected communities about how the measurement was produced

The big idea: the biggest model in the world cannot fix a bent ruler. Choosing the right thing to measure is often more important than choosing the right algorithm.