What Should an Agent Remember?

When Nicole Butterfield, executive editor at O’Reilly, and I started planning Expert Showcase: Agent Memory, the conversation kept circling back to how complex the topic gets once you start looking into it. We wanted to explore it from several angles, from an agent learning through experience to a robot interacting with the physical world, and an organization whose knowledge still lives in people’s heads (a recurring problem we talked about in The Org Age of AI).

When I first covered agent memory at Turing Post, I used the movie Memento as an analogy: trying to make sense of the world through notes and photographs because you can’t form new memories. It captures a frustration many of us have had with AI.

But preparing this event made me think further: once an agent can remember, what does it unlock for us, what becomes possible? That’s what I’m bringing to you today: a structured way to look at agent memory with the people who spend their days building it. Nothing here is solved or set in stone, which is precisely why it is so fascinating and worth exploring.

In the previous AI Builds AI episode, we explored skills: reusable procedures that help agents do their work. Memory takes us a step further back, to where those procedures might come from: an agent gets corrected, tries something that fails, or discovers a better way through a task. How that experience becomes meaningful, and what it takes to keep it useful as circumstances change, is what we’re going to explore today.

To put it simply, memory gives an agent a way to use information from earlier work in a later decision. That information might be a preference, a fact about a project, a record of a failed attempt, or a procedure that worked well. For it to help across separate sessions, the surrounding system needs to retain it and make it available when the agent needs it again.

This is where what we mean by “memory” makes it confusing for us. Because when we remember something, we rarely think about which part of the process stored it, found it, or connected it to the present situation. With agents, someone has to design those steps. A larger context window can hold more material during a task, but it does not decide what deserves to survive afterward, or how a future agent should use it.

Lewis Liu, who works on enterprise AI at Microsoft, offered an interesting way to look at this problem by separating three ambitions: personalization, self-improvement, and organizational memory. They often appear in the same conversation, although each asks something different of the system.

Personalization helps an agent adapt to a particular person: how you like your reports, which projects you work on, what you have already explained. Self-improvement asks whether experience can change how the agent performs a task. Organizational memory extends the question beyond one user or agent: can knowledge accumulated across a company become available to others who need it?

These three ambitions ask different things of memory. Here’s how we might recognize whether each is working.

The same interaction could contribute to all three, but success at one does not establish success at the others. Remembering my preferred format tells us very little about whether an agent can learn a reliable method, let alone whether a company should adopt it. Once we make these ambitions explicit, we can ask a more useful question: what should be different the next time the agent does its work?

Don’t settle for shallow articles. Go deeper with us. Find inspiration for what to build and the knowledge to put it into practice.

Join Premium members from top companies like Microsoft, Nvidia, Google, Hugging Face, OpenAI, a16z, plus AI labs such as Ai2, MIT, Berkeley, .gov, and thousands of others to really understand what’s going on in AI.

AI agent memory is information an agent stores and retrieves to use in later work, such as facts, preferences, past experiences, or procedures. Persistent memory survives beyond the current conversation. Its usefulness depends on what the system keeps, when it retrieves it, and whether it can correct or remove information that no longer applies.

A context window holds the information a model can use in its current response. Retrieval-augmented generation, or RAG, brings selected external information into that context. Agent memory can use retrieval too, while also retaining and updating information from earlier interactions or activity. Moving information out of context does not by itself make it persistent across sessions.

Episodic memory records earlier experiences or interactions, semantic memory stores facts, and procedural memory captures instructions or methods for doing a task. In Pete Johnson’s architecture, these appear as interaction summaries, relevant facts, and reusable plans, alongside taxonomic memory for domain terminology. The categories help determine what to retrieve for a particular job.

Organizational memory makes a company’s accumulated knowledge and experience available across agents and teams. It can include approved exceptions, procedures, terminology, and the reasons behind decisions. Useful shared memory preserves sources, scope, and change history, with clear ownership and permissions so that teams can question, correct, and safely reuse what others have learned.

Compare the same kinds of tasks with and without memory, using fresh sessions and tasks beyond the original correction. Check actual outcomes, repeated errors, and total cost, including memory writing and retrieval. Repeat trials and test changed circumstances, conflicting evidence, and stale information. A useful memory improves later work without spreading an earlier mistake into new situations.

⬅️ Previous in AI Builds AI: Skill Up: How AI Builds the Skills That Improves Other AI

🏗 Explore the full AI Builds AI series