Why must test data be kept separate from training data?

So the test shows how the model does on new examples

Because test data is secret code

Course 4, lesson 35 of 100, Ages 10+

Training and testing

Practice, then the quiz

Like I’m 5

First you practise spellings with the answers in front of you. Then your teacher tests you on new words. AI learns the same way.

The big idea

To know if a model really learned, we test it on examples it has never seen. So data is split into a training set for practice and a test set for the final exam.

If we tested on practice examples, the model could look brilliant just by remembering them. A fair test on fresh data tells us how it will do in the real world, with new photos, new voices or new emails.

Examples

School exams: A good exam uses new questions, not the exact homework.
Face unlock: Tested on photos of people the model didn't train on.
Leaky tests: If test examples sneak into training, the score looks better than it really is.

How it works

AI learns in two stages. First it practises on examples where it can see the answers. That’s training.
Then it takes a quiz on examples it has never seen. That’s testing.
If it only does well in practice, it memorised instead of learning. Just like cramming for a test.

Check your understanding

Why test AI on examples it has never seen?: Options: To check it really learned; To make it slower; Because practice is boring.
Answer: To check it really learned. New examples show whether the AI learned the idea, or just memorised the answers.
Why must test data be kept separate from training data?: Options: So the test shows how the model does on new examples; To save space; Because test data is secret code.
Answer: So the test shows how the model does on new examples. Fresh test data reveals real-world performance, not memory.

Remember

Train on practice examples, then test on new ones.

Talk about it

Why is a surprise quiz a good test of real learning?

Go deeper

Data is split into training, validation and test sets. A model that scores well in training but poorly on new data is overfitting. Measures like accuracy, precision and recall track performance, and the test set stays untouched until the end so the final score is honest.