Next-Word Prediction

Play the game yourself. "The dog chased the ___". Before you finished reading, you had a shortlist: cat, ball, boy. You also had a feel for the odds. Cat is the favourite. Car is possible but unlikely. Parliament is not on the list. You did this instantly, without deciding to. A language model does the same thing, except that it puts an explicit number on every word it knows, and does it at every position in the text.

Now change one word. "The detective chased the ___". Your whole shortlist just rewrote itself: suspect, lead, thief. Change it again. "The toddler chased the ___". Ball, bubbles, dog. Nothing about the word "chased" changed, yet the predictions moved completely, because everything that came before them moved. Try "The ___ chased the cat" and notice you are now guessing backwards from the object.

So the prediction is never about one word. It is about everything that came before it, and a small change anywhere upstream can reshuffle the whole list. That is the point to take from this chapter: predicting the next word means reading the context, all of it, every time. Keep the shortlist game in mind. The next few chapters are about what it takes to play it well, and what playing it well seems to require.