Chatterfly

Chapter 13 of 36 · ~1 min

Memory

The loop's state is the model's short-term memory: whatever is in its context window right now. It is large but finite, and it disappears when the run ends. Anything an agent should know across runs (a person's preferences, the outcome of last week's task, a fact it looked up yesterday) has to be stored somewhere and brought back deliberately.

Memory is therefore a design problem, not a feature you switch on. What should be remembered, for how long, for whom, and who may see it? Remembering too little makes an agent forgetful. Remembering too much makes it expensive, confusing, and a privacy liability.

Memory layers around the modelPersistent memoryacross runs: preferences, past outcomes, facts looked up earlierRun statethis run: the inputs and outputs of every step, kept until the run endsContext windowwhat the model sees right now: large, finite, gone when the turn endsinstructions · conversation so far · retrieved passages · tool resultsRetrievalsearch, then insertbrought back deliberatelynothing reaches the modelunless something puts it here
What the model sees now, what the run keeps, and what survives across runs.

Experiment

Live model

Tell the model something, then ask about it later, with and without the earlier turns in its context.

Turn 1: tell the model something

Turn 2: ask about it

Runs against a live model through Chatterfly's server. Your text is sent to the model provider and not stored. Nothing is remembered between your clicks except what the page sends back each time.

Big question

What should an agent be allowed to forget?