Bias in AI

AI learns from human-made data. Human-made data is full of human biases — things we got wrong, things that were unfair. When the AI learns from that data, it learns the biases too.

Famous examples

• An early resume-screening AI at a big tech company learned to rate resumes with women’s names lower — because it had been trained on historical hires that skewed male.
• Early image-generation models drew “a doctor” as a white man and “a nurse” as a white woman, every time. They had learned those associations from photo captions online.
• Speech-recognition models have historically been worse at understanding Black American English — because the training audio didn’t represent it.

Three kinds of bias

1. Representation bias.The training data doesn’t include everyone equally.
2. Measurement bias. The labels humans added are themselves unfair.
3. Aggregation bias.The AI learns one rule for “everyone” when different groups need different rules.

What companies do about it

Every major AI company now tests their models for bias before release. They hire red-teamers— people whose job is to try to get the AI to do something unfair — and fix what’s found. They’re not perfect, but the problem is visible now in a way it wasn’t five years ago.

What you can do

When you use an AI to make decisions about real people, stop and ask: what’s the bias hiding in its training data? Would this answer change if the person it’s about were a different gender, race, age, or language? If you don’t know, don’t trust the answer.