How can builders find out whether an AI is unfair?

Test how it performs for different groups

Course 8, lesson 71 of 100, Ages 11+

Fair or unfair?

How bias sneaks in

Like I’m 5

If you only ever saw red apples, you might think green apples aren’t apples. AI makes unfair mistakes like that when its examples leave things out.

The big idea

AI can be unfair when its training data leaves people out or reflects old prejudices. A hiring tool trained on past hires from one group may score that group higher, repeating the bias.

Fairness takes work: checking how a model performs for different groups, fixing gaps in the data, and letting people question decisions. Fairness also means asking whether AI should be used for a decision at all.

Examples

Face recognition: Some systems made more errors on darker skin tones because of unbalanced data.
Hiring: A CV-screening tool learned to prefer one gender from past data.
Fixing it: Testing results separately for each group, then improving the data.

How it works

AI learns from data made by people. If that data is unfair, the AI can be unfair too.
Imagine an AI that only ever saw pictures of doctors who were men. It might wrongly guess a woman can’t be a doctor.
Fair AI needs data that includes everyone, and people who check the results.

Check your understanding

Where does AI bias usually come from?: Options: The data it learned from; The colour of the computer; The weather.
Answer: The data it learned from. AI copies the patterns in its data, including unfair ones.
How can builders find out whether an AI is unfair?: Options: Test how it performs for different groups; Ask it if it's fair; Use it more quickly.
Answer: Test how it performs for different groups. Comparing results across groups reveals hidden bias.

Remember

AI can copy unfairness from its data, so people must check.

Talk about it

When is a rule fair for everyone?

Go deeper

Bias can enter through unrepresentative data, labels that reflect human prejudice, or goals that ignore some groups. Responsible teams measure results across groups, rebalance data and audit systems used for high-stakes decisions such as hiring, lending or healthcare.