What Is a Language Model?

Here is the whole trick. A language model looks at some text and guesses what comes next. That is it. It does not look things up, run a program, or hold beliefs. It has been shaped by an enormous amount of text so that its guess about "what comes next" matches the patterns in what people have written. Then it guesses again. And again.

Now hold that next to what you have seen these systems do. Write working code. Summarise a contract. Explain a joke. Argue both sides of a question. Translate, plan, negotiate, tutor. All of it comes out of one small operation, repeated. Nobody designed a module for reasoning or a module for humour. They emerged, and the people who built the models were as surprised as anyone.

That is the fascination this course is built on, and the question it keeps asking: how does so much come out of so little? We will follow the thread from a single predicted word to systems that use tools, carry out long tasks, and work alongside people. At each step you should be able to see the next-word predictor still there underneath, doing what it always does. That view is what lets you see clearly both what these systems can do and where they stop. The capability is real. So are the gaps, and they explain most of what goes wrong when models are put to work.