Course 6, lesson 51 of 100, Ages 12+
How chatbots write
Guess the next word
Like I’m 5
A chatbot is like a friend who has read every book in a giant library and loves the game ‘what word comes next?’
The big idea
A large language model (LLM) learns by reading enormous amounts of text and practising one game: guess the next word. After billions of rounds, it becomes very good at continuing any text.
Because the training text includes stories, explanations and conversations, the model picks up grammar, facts and styles along the way. When you chat with it, it's still predicting what comes next, just in a very sophisticated way.
Examples
- Autocomplete: Your phone guesses the next word too, but with a much smaller model.
- Writing help: An LLM can draft an email when you describe what you want.
- Explaining: Ask it to explain photosynthesis like you're ten, and it adjusts its style.
Check your understanding
- What game does an LLM keep playing?
- Options: Guess the next word; Hide and seek; Snakes and ladders.
Answer: Guess the next word. An LLM predicts the next token again and again to build its reply. - What game does an LLM practise during training?
- Options: Guessing the next word; Hide and seek; Chess.
Answer: Guessing the next word. Next-word prediction, repeated billions of times, is the core of LLM training.
Remember
LLMs write by predicting one token at a time.
Talk about it
Can you finish this like an LLM: “Once upon a…”?
Go deeper
LLMs predict the next token (a word or part of a word) using probabilities learned in training. The creativity setting, called temperature, controls how adventurous the picks are. Because it’s prediction, not lookup, answers can sound confident and still be wrong.