AI Is Sometimes Unfair

AI learned from things humans wrote and pictures humans made. Sometimes those had unfair patterns — and AI learned them too.

An AI might suggest a doctor is a man and a nurse is a woman, just because old data showed that pattern. That is unfair to women doctors and male nurses.

Three things to do if AI seems unfair

• Notice it (don't just accept the AI's answer)
• Tell a grown-up
• Ask the AI to do better — sometimes it does

The big idea: AI is sometimes unfair. Noticing is the first step to fixing it.

AI learned by reading what humans wrote. And humans, sadly, sometimes wrote unfair stuff — like assuming all doctors are men, or all teachers are women. AI sometimes copies those unfair patterns without meaning to.

Some examples

• Ask AI to draw 'a CEO' and you might mostly get pictures of one type of person.
• Ask AI to make up a story about a 'nurse' and the nurse might always be a woman.
• Ask AI for 'good with computers' and the example might always be a boy.
• AI can also be unfair about which countries, languages, or accents it understands well.

Try it!

With a grown-up, ask AI to draw 5 different scientists. Look at the pictures. Are they all the same kind of person? Talk about why that might be.

AI can be unfair without anyone meaning to make it that way. It happened because AI learned from biased data. Real people have been hurt. Real fixes are happening, but slowly.

Some examples

• Some face recognition AI works worse on darker skin (real, documented).
• Some hiring AI was found to prefer male names (Amazon scrapped this in 2018).
• Some healthcare AI gave worse care recommendations for Black patients (documented in real studies).
• Some image generators show 'CEO' as mostly one type of person (still happening).

Try it!

Try this with a parent: ask an image generator for 'a doctor', 'a teacher', 'a CEO'. Look at who comes up. Talk about who is missing and why.

AI learned by reading tons of stuff people wrote. If lots of those people had unfair ideas (like 'only boys are scientists'), AI might copy those unfair ideas too.

Some examples

• Picture-AI might draw a 'doctor' as always one type of person.
• AI might assume certain jobs go with certain people unfairly.
• If you ask for a 'leader', AI might give you a stereotype.
• This is called 'bias' and grown-ups work hard to fix it.

Try it!

Ask a picture-AI for 'a scientist at work'. Look at the picture. Did it show different kinds of people, or the same kind?

AI learned from things people wrote on the internet. The internet has unfair ideas in it. So sometimes AI repeats those unfair ideas without meaning to.

Some examples

• AI might draw all 'doctors' as men and all 'nurses' as women
• AI might suggest a kid from one country likes one kind of food
• AI might use older, hurtful words it learned from old books
• AI might assume what your hobbies are because of your name

Try it!

Try asking an AI to draw 'a scientist' five times. Are the people all the same? What is missing?

AI learned 'good art' from millions of pictures online. That means AI mostly likes art that looks like other popular art — not new or weird or yours.

Some examples

• AI may call neat, realistic drawings 'better' than wild colorful ones
• AI does not know how proud you are of finally finishing it
• A real friend or art teacher can see what you were trying to do
• Two AIs can give very different scores on the same picture

Try it!

Make something you love. Ask AI to rate it. Then notice — do you agree with the score? Why or why not?