Chatterfly

Chapter 8 of 36 · ~1 min

How ChatGPT Happened

A raw language model completes text. Ask it a question and it may continue with more questions, because that is what often follows a question in its training data. Turning a predictor into an assistant took extra steps: tuning on examples of instructions and good responses, further tuning on human preferences between candidate answers, and then a system prompt and a chat interface around the whole thing.

None of this changed what the model fundamentally does. It changed the distribution of what it predicts, so that "what comes next" after a question is now, usually, a helpful answer. Keep this in mind whenever an assistant seems to have a personality or intentions.

How a predictor becomes an assistantRaw language modelcompletes textInstruction tuningexamples of good answersPreference tuninghuman picks between answersSystem prompt + chatthe interface layerThe same predictor underneath at every stage. What changed is the distribution of what comes next.
From a text completer to an assistant: three layers on top of the same predictor.

Experiment

Live model

Ask one question. See the raw model's continuation, the assistant's answer, and the same assistant under a different system prompt.

Alternative system prompt:

Runs against a live model through Chatterfly's server. Your text is sent to the model provider and not stored. The raw column uses a completion model; the other two use a chat model.

Big question

What exactly changed between the predictor and the assistant, and what did not?