Tendril
Loading lesson…
Even accurate data can encode an unjust history. The COMPAS recidivism tool shows what happens when AI learns from a biased past.
COMPAS is a commercial risk-assessment tool used by US courts to predict whether a defendant will re-offend. In 2016, ProPublica published an investigation showing that COMPAS was nearly twice as likely to falsely flag Black defendants as high-risk compared to white defendants. This case became the most famous example of historical bias in AI.
| Error type | Black defendants | White defendants |
|---|---|---|
| Falsely labeled high-risk (didn't re-offend) | 45% | 23% |
| Falsely labeled low-risk (did re-offend) | 28% | 48% |
| Overall accuracy | ~65% | ~65% |
COMPAS was trained on historical data: arrests, convictions, re-arrests. But the US criminal justice system has a well-documented history of policing Black neighborhoods more heavily, leading to more arrests for equivalent behavior. The model learned to replicate that historical pattern, not because the programmers intended bias, but because the data itself reflected generations of unequal enforcement.
In 2016, Wisconsin's supreme court ruled that COMPAS could be used in sentencing, but only with warnings about its limitations. The debate continues to this day, and COMPAS remains in use in several US jurisdictions.
The big idea: a model trained on an unjust past will perpetuate that injustice into the future. Technical accuracy is not a defense. AI used in high-stakes decisions demands moral choices, not just statistical ones.
15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-data-historical-bias-compas
What is the core idea behind "Historical Bias: The COMPAS Case Study"?
Which term best describes a foundational idea in "Historical Bias: The COMPAS Case Study"?
A learner studying Historical Bias: The COMPAS Case Study would need to understand which concept?
Which of these is directly relevant to Historical Bias: The COMPAS Case Study?
Which of the following is a key point about Historical Bias: The COMPAS Case Study?
Which of these does NOT belong in a discussion of Historical Bias: The COMPAS Case Study?
What is the key insight about "The paradox" in the context of Historical Bias: The COMPAS Case Study?
What is the key insight about "A hard mathematical result" in the context of Historical Bias: The COMPAS Case Study?
What is the recommended tip about "Ground your practice in fundamentals" in the context of Historical Bias: The COMPAS Case Study?
Which statement accurately describes an aspect of Historical Bias: The COMPAS Case Study?
What does working with Historical Bias: The COMPAS Case Study typically involve?
Which of the following is true about Historical Bias: The COMPAS Case Study?
Which best describes the scope of "Historical Bias: The COMPAS Case Study"?
Which section heading best belongs in a lesson about Historical Bias: The COMPAS Case Study?
Which section heading best belongs in a lesson about Historical Bias: The COMPAS Case Study?