Debiasing: What Actually Works and What Does Not

For a decade, debiasing has been a cottage industry in ML research. Dozens of techniques promise to remove bias from word embeddings, face recognition, or classifiers. A 2019 paper, Lipstick on a Pig by Gonen and Goldberg, showed that many word-embedding debiasing methods just hid the bias without removing it. A cluster analysis could recover the gender signal.

Three places to intervene

What tends to work

• Collecting more diverse data (the single most effective intervention)
• Re-weighting training examples to equalize subgroup representation
• Setting different decision thresholds per group to equalize false-positive rates
• Explicitly measuring and reporting per-group performance
• Community review of deployment plans

What often does not

• Removing protected attributes from training data (correlated features leak the signal anyway)
• Adversarial debiasing (often unstable, can collapse to trivial solutions)
• Post-hoc fairness metrics without root-cause analysis
• One-shot debiasing claims that do not hold up on new distributions

The impossibility problem

A realistic debiasing playbook

1. 1Identify the harm you care about most
2. 2Pick a fairness metric aligned with that harm
3. 3Collect more representative data first
4. 4Apply reweighting during training
5. 5Measure disaggregated performance before and after
6. 6Use threshold tuning to reach equalized error rates
7. 7Disclose residual bias honestly in the data card

The big idea: there is no silver bullet for bias. What works is a combination of better data, honest measurement, thoughtful trade-offs, and humility about what algorithms can accomplish.