This is Episode 1 of Memory, Forgetting, and the Self in the Machine: how intelligence reaches across time when it cannot carry the whole past with it.
Picture a Tuesday. You run a small catering business, and you spend twenty minutes with an AI assistant working out delivery windows. It is good at this. It knows that a lunch order should leave the kitchen with time to spare, that a refrigerated van changes what you can promise in August, and that customers who are told “between eleven and one” hear “eleven”. Somewhere in the conversation you mention, once, that the bridge on the north side is closed until spring, so nothing can go north before ten. The assistant takes it in and builds the schedule around it.
The next Tuesday you open a new conversation and ask for help with the week’s routes. The assistant is just as fluent. It proposes a nine o’clock run to the north side. It is an expert on delivery windows and meets you, every morning, as someone it has never met. It has not misplaced the bridge. Nothing in it ever had the bridge to lose. The assistant has a great deal of knowledge and no past with you. Those are different properties, and the difference gives you a question to ask of any system that claims to remember you: by which door did the past come in?
For the argument to be clean, fix the example. The assistant is a released model with no memory feature switched on, and it runs on the same internal settings on both Tuesdays. Some products change those conditions. For now, the question is what would have to cross from one Tuesday to the next for the bridge to matter, and where it could cross.
What it does know
Start with the fluency, because it is real and it is the source of the confusion. In 2019 a group of researchers led by Fabio Petroni asked a plain question about language models: how much of what they learned during training can be read back out, simply by asking? Models of this kind are trained to fill in missing words. The paper’s own example is a sentence with a gap: “Dante was born in [Mask] in the year 1265.” Feed a version of that sentence to the model, and it fills the gap with “Florence”.
The model did not need a table of birthplaces to answer. It read an enormous amount of text during training, and that text left its mark in the model’s parameters: a very large set of numbers, adjusted again and again during training until the model got good at predicting the missing word. After training, for a released model like the one in our example, those numbers stop changing. Petroni’s team found that without any further training, the model BERT held “relational knowledge competitive with traditional NLP methods that have some access to oracle knowledge”, meaning systems given some access to correct information in structured form. In the paper’s first figure, the language model is drawn under the label “Memory Access”. The field’s own picture calls this memory.
Notice the paper’s wording, though, because it is careful in a way the figure is not. “The parameters of these models appear to store vast amounts of linguistic knowledge.” Appear to store. The researchers measured what came out when they asked; they did not open the model and point to the place where Florence sits, and the paper makes no claim about where inside the numbers a fact lives. They even noted that rephrasing a question changed the answers, so what they measured was “a lower bound for what language models know.” The knowledge is real in the sense that matters for using the model: ask, and it answers correctly. Where it sits is something we do not know.
Two features of this knowledge matter for Tuesday. First, it is general. It came from training text and it is the same for everyone who uses the model; your question about Dante and a stranger’s question about Dante meet the same numbers. Second, it was set before you arrived. In our example, nothing you typed last week went into training, and the numbers on the second Tuesday are identical to the numbers on the first. Researchers sometimes call this route “parametric memory”, and the name is fair as far as it goes: it is how what happened in training reaches every later answer. It is not a record of any conversation, and the bridge is not in it.
What it is handed
The second route is simpler to see and easier to underrate. Every time the model produces an answer, it works from what it is given at that moment. In a chat, that is your latest message, the earlier turns of the same conversation, and any instructions the product adds behind the scenes. This is the context: whatever is in front of the model for this computation.
The mechanism is described in the 2017 paper that introduced the Transformer, the design underneath most current assistants. That paper was about translation, not chat, but the step it describes carries over. The model “generates an output sequence ... one element at a time. At each step the model is auto-regressive, consuming the previously generated symbols as additional input when generating the next.” Read it slowly and it says something modest and decisive. The model produces a word, adds that word to what it was given, and produces the next word from the longer sequence. Its working material at each step is the sequence in front of it. What is not in the sequence does not enter the computation.
That is why, inside a single conversation, the assistant seemed to remember the bridge. You said it in the third message; by the tenth, your third message was still part of the sequence the model was handed at every step. It did not have to recall the bridge. The bridge was in the input.
The context can do a surprising amount. When OpenAI introduced GPT-3 in 2020, the headline finding was that a model could pick up a new task from a few examples placed in front of it, with, in the paper’s words, “no weight updates”, meaning the parameters did not change at all. The authors called this in-context learning. They were also frank that it was unclear whether the model was learning anything new “from scratch” or recognising a task it had already met in training, so the safe claim is narrower than the name: what is in the context changes what the model does, without changing the model. The context also has an edge. GPT-3 could take in 2,048 tokens at once (a token is a small piece of text, often a word or part of one); every model has some limit, and the context exists only for the computation it feeds.
Which brings us back to Tuesday. A new conversation starts with a new context. At least one vendor says so in its own documentation: Anthropic’s page for Claude Code, as it read on 3 November 2026, opens its account of memory with “Each Claude Code session begins with a fresh context window.” Last week’s sequence is not in this week’s input unless something puts it there.
The two runs
Take the second Tuesday twice. In the first run, last week’s conversation happened. In the second, imagine it never did. What does the model receive in each case? The same parameters, because in our example nothing changed them. And an input with nothing from last week in it, because the new conversation contains only what you typed this morning and whatever the product adds behind the scenes. Same numbers, and no trace of last week in the input: the computation has nothing in it that carries last week. Whatever the assistant says about the north side, last week cannot be the cause. Answers may vary from one run to the next, because these systems often pick among likely words with some randomness, but that variation carries no information about last week. It would vary the same way if last week had never happened.
So, for a model like the one in our example, an earlier moment can shape a later answer in two ways only. It can change what lasts in the model, which for a released model means what training left in the parameters. Or it can be present in what the model is handed now, the context. Your conversation about the bridge did neither. It was not part of training, and it is not in this morning’s input. It had no door.
This also tells you what will work, and you can check it yourself in an assistant like the one in our example. Copy last week’s conversation and paste it into this week’s. The bridge is back in the input, so the answer can now take it into account, and the nine o’clock run north should go. That is not the model remembering. It is the past being handed back through the second door, so that it becomes part of the input. Then try the opposite: ask the same fresh question on two different days, with nothing pasted in. You will get competent answers both times, and on neither day will the assistant know you better than it did the day before. Repetition does not open the first door; only training does.
It helps, at this point, to look at what the word “memory” has been naming. In the papers above it is used three ways for two different things. Petroni’s figure labels the knowledge in the parameters “Memory Access”. The 2020 paper by Patrick Lewis and colleagues that introduced retrieval-augmented generation calls the same parameters “parametric memory”. The 2017 Transformer paper uses “memory keys and values” for something else again: the model’s internal summary of the sentence it is translating right now, which it consults while writing the translation. Same word, three usages, two referents. The first two are what training left; the third is a working product of the present input. None of them is a record of the system’s own earlier conversations. The researchers are not being careless; within each paper the meaning is clear. The trouble starts when the word travels without its meaning, and a reader hears “the model has memory” as “the model has a past with me.”
Where this stops
Everything above rests on the conditions we fixed at the start: a released model, parameters unchanged between the two Tuesdays, no memory feature. Real products are free to change those conditions. A company might retrain on user conversations, which would reopen the first door slowly and for everyone at once, or it might build in a mechanism that carries notes from one session to the next. Whether a particular product does either is a question about that product. The Claude Code documentation quoted above, for instance, goes on to name two such mechanisms for carrying knowledge across sessions. That is the point to notice: the fresh start is the default, and continuity is something added.
So the idea to take away is a distinction rather than a fact about any assistant. Knowing a lot and having a past are different properties. The first comes from training and is shared by everyone who talks to the model. The second requires that something from your earlier moment actually reach this one, and for a fixed model the only two routes we have examined are a change in the parameters and a presence in the context. When a system seems to remember you, ask which door the past came through. When it forgets you, ask which door was closed.
Two doors are not the whole house. A process that keeps running can carry something of its own from one step to the next, and a record can be written down outside the model and fetched later. Those are two more places a past can hide.
This is Memory, Forgetting, and the Self in the Machine: how intelligence reaches across time when it cannot carry the whole past with it.