What Does an AI Agent Actually Remember? A Plain-English Guide
You spend an hour with an AI assistant working through a project. The next morning you open a new chat, say "let's pick up where we left off," and get a polite blank stare. Then, a week later, the same assistant casually mentions your dog's name. Which is it: does it remember you or not?
The answer is "both, in different ways." An AI agent doesn't have one memory. It has three, and they behave very differently. Once you can tell them apart, most of the strange forgetting (and the occasional surprising recall) starts to make sense.
Layer one: the context window, or short-term memory
Everything an AI model "knows" about your current conversation sits in its context window: the block of text it reads before writing each reply. Your messages, its answers, any document you pasted in, all of it goes there.
Two things about the context window surprise people.
First, it is wiped clean when the conversation ends. The model itself doesn't learn from your chat. Start a new thread and, unless something else steps in, you are talking to a stranger.
Second, even inside one conversation, attention isn't even. A well-known Stanford-led study, Lost in the Middle, found that language models are best at using information placed at the very beginning or very end of a long input, and noticeably worse at using details buried in the middle. That's why a long chat can start to "drift": the instruction you gave forty messages ago is still technically there, just harder for the model to act on.
Practical tip: if a detail matters, restate it near the end of your message rather than trusting it from earlier.
Layer two: saved memory, or notes about you
To bridge the gap between conversations, many assistants now keep notes. ChatGPT, for example, splits this into two kinds: "saved memories" you explicitly ask it to keep (your name, how you like answers formatted) and "chat history" insights it picks up from past conversations on its own. Other assistants offer similar features under different names.
The key thing to understand is that these are notes, not recordings. The assistant isn't rereading last week's conversation word for word. It's consulting a short summary of things it decided were worth keeping. That explains the dog's-name moment: a small personal fact made it into the notes, while the detailed project plan you spent an hour on did not.
It also means memory can be wrong or out of date, and you can usually fix it. OpenAI's help pages note that saved memories can be viewed and deleted individually, and that turning off chat-history references schedules that information for deletion. Check your assistant's settings occasionally; it's worth knowing what it thinks it knows about you.
Layer three: connected data, or the filing cabinet
The third layer is the one that matters most for agents doing real work: the data they can look up. When an agent is connected to your CRM, helpdesk, calendar, or shared drive, it doesn't need to "remember" a customer's order history. It fetches it, fresh, each time it's needed.
This is the most reliable kind of memory because it isn't memory at all. The source of truth lives in your systems, and the agent reads from it. It's why a support agent plugged into a ticketing system can greet a returning customer with context, while a general-purpose chatbot with no connections cannot.
The trade-off is access. An agent only knows what it's been allowed to see. If it seems clueless about something, the first question isn't "is it dumb?" but "is it connected to where that information lives?"
Why your agent "forgot" yesterday
Put the three layers together and the mystery mostly dissolves. Yesterday's conversation lived in the context window, which is gone. Unless a detail was written to saved memory, or stored in a system the agent can query, there's nothing for it to recall.
So the fix is usually simple.
For one-off continuity, paste a short summary of where you left off at the start of a new chat.
For lasting preferences, tell the assistant explicitly to remember them, then check that it did.
For business work, don't rely on chat memory at all. Put the important information in a document, ticket, or record the agent can read.
The takeaway: stop thinking of AI memory as one thing that either works or doesn't. Think of it as a whiteboard, a notebook, and a filing cabinet. Decide which one each piece of information belongs in, and your agent will seem a lot smarter overnight.
Sources: Liu et al., "Lost in the Middle: How Language Models Use Long Contexts" (TACL, 2023) · OpenAI Help Center — Memory in ChatGPT · OpenAI — Memory and new controls for ChatGPT