Why Agents Need Memory

Introduction and Experience

Hey, we will talk about the agents and why agents need memory, and how it could look like, and why, for example, RAG is not enough and why RAG is not true memory and all of this thing. So, you know, I was working on the memory for the last two years. I was working on it actively when I was the part of the Keen, the privacy-focused AI assistant. Right now, I am working more on the agents and the protocols and agent identity at Affinity, but still memory is my big passion.

The Landscape of Agent Memory

If you’re following the space right now, we have the huge booming topics like context graphs, decision traces, how to make agents learn from his decisions. We also have some manipulations around this because we dehumanize the agents and actually compare them with the people that could take ad hoc decisions, and actually for me, it’s not really true.

Anatomy of an Agent

But you know, let’s talk about the agent anatomy first. So majority of the agents that we’re talking about nowadays, unfortunately in a core have the large language models. And large language models have some kind of memory because they have the training data stored in weights, and actually the LLM has something that it is remember, and it remembers a lot. But the problem is that it’s not acquiring new information and it’s not acquiring information, for example, user gives to them or some company specific if we talk about the enterprises.

The Rise and Limitations of RAG

And then what we do, we build the knowledge base and we apply the RAG, yeah, so augmented generation. And this helped us to bring the data that was not in the data set and allow LLM to work with them. And actually RAG was quite successful and we start from simple vector databases and vector search, then we all realize that it gives us some decent results, works quite well, for example, for the documents, works quite well by the way if we have the limited amount of documents. Because if the amount of information will grow, we will see the decay, and especially when we build the vector space representation of all our documents and we will see if they condense. So if they focus on one topic in one place or they just sparse and distributed. The more condensed they are, the more the errors will get because we do not know which of the chance to choose.

Advanced Memory Strategies

And then we apply different strategies. It was the boom of graphs, entity recognition, different strategies combining the RAGs with the graph with the vector search for RAG, then combining the graphs with the vector search for RAG and some advanced topics from naive BM25 indexing to some advanced graph neural network indexes and all of the things.

And all this helped, but still the accuracy struggle, and we even have the research that it was the broken promise that actually the RAG do not solve the hallucination of the agents, it’s improve a bit. And then we stay with the questions, okay, if not RAG, what we could do to fight with the hallucination from one side, from another side improve the conversation experience.

Shifting from Bots to Agents

And right now when we shifted from the bots that talk with the users to the agents that do something for the users, we discovered that we have the new properties to the agents that not just needs to maintain the value conversation and pick up the facts that relevant to the conversation. We have much bigger challenges: how the agent could follow the plan, how the agent could execute the tasks without the hallucinations, and how to make his actions align with our goals and wishes. And it’s way more challenging task than just simply correlation of the information chunks that GraphRAG gives us.

Mimicking Human Memory

So then we end up with the concept of the memory because we trying to mimic the humans and we have the memory. And it’s interesting and deep talk: how human memory works, do we have one memory or we have multiple memories, how it’s organized and how we do the processing.

But if we could shortly talk about the memory and why RAG is not memory, the first big thing, especially for the conversational memory, that the RAG doesn’t have the context. The RAG doesn’t have the deep context in episodic context.

So we remember something in a context of the emotions and events that happened to us. And we don’t have the emotional context, so we don’t have the emotions at all. Right, we could capture them somehow, but majority of the systems doesn’t have even the notice of this, right? And the human memory, the stronger it is, the stronger the emotion was. And if they’re negative or they’re just dangerous stuff to you, you will remember it much better. I don’t think that it’s a positive of the human memory, but it was the survival mechanism.

Episodic Memory and Processing

And another one that these events or the information that we remember more focused on the episodic memory. And we even have the episodic buffer. So we have the big chunks that have the episodes, they go to the episodic buffer, then they go to the temporal memory, then in night we just process the memories, move them to the permanent memory, and then we have a couple of cognitive processes that do the semantization of the episodic memory. And episodic memory decay faster, but all our memory, somehow the big chunk of our declarative memory actually is the episodic memory and have the fight episodic context.

The Challenge of Time in Memory

And unfortunately RAGs struggle with it. Yeah, we could build some graph-based memory systems that have the context concept, that have the episodic concept as event graphs, for example, do. Then we get completely new challenge that we wants to process the episodic memory and now we need the notion of time. And time itself is a huge, huge topic.

And actually I wrote a book about the AI memory and time because time deserve itself the book, and maybe two to be honest. So this episodic context go quite tight to the understanding of time, processing time, and actually time is not timestamps. Time actually the sequence or the DAGs, the directed acyclic graphs of the events.

And if we’re talking about the human, these events have some different emotions and different impact on our life. They have some consequences, they have the relation, they have the casual relation between each other. And I talking about this architecture of this event-focused, casual, time-aware memory in my book. So it will take a quite long time to describe it here, but if you wants to understand it, you could read my book about conversational memory or you could also find the architecture of this memory in the my new book that I still working on it.

https://leanpub.com/time-aware-ai-memory

Beyond Context Graphs

It’s a book about what’s goes beyond the context graph and how to make the agents actually do what we ask them to do. And another thing that GraphRAG have quite limited capabilities to build the associations and the semantic packages. So we always remember the semantic information in a connection to another semantical concept. That’s why we all like graphs because they somehow mimic how our semantic memory works because our semantic memory is the huge network.

But the trick is that our semantic memory, if we read some researches, it’s not the directed graph. It’s more the multi-layered hierarchical network that have at least three kind of hierarchies and some sub-hierarchies and the classification of the things. So you have some mixture of multi-layered graph that form this complex network. But our main ability is to build the associations and build the links between the concepts, and yeah graphs could do it, but GraphRAG even have the quite limited capacities to do it.

https://leanpub.com/beyondcontextgraphs

Memory as a Cognitive Process

And the biggest part that RAG is only one cognitive process, right? That you get the question, you find what’s relevant, you give it back. But human memory have huge amount of cognitive processes: of processing, extracting, memorizing, semantization, the dynamic recall, the decay, and forgetting.

And forgetting itself actually is the big topic that we will talk next. But your retrieval happens without the understanding of the information, it’s just some mathematical filtering like BM25 or some graph algorithms that find the centrality of the different entities and all other thing. It’s too naive and it’s not enough.

The Role of Forgetting and Recall

And I already mentioned that the memory require forgetting and forgetting is a huge, huge topic because our memory is more focused to forget. But it’s still big question how we forget, do we forget forever or we just change the priority and importance of this information and we have struggle to recall it and all of the things.

And recall actually have the loop: the more often you recall the information, the better you remember it, you make this neural network stronger and you just remind that it’s something important to you and all other thing. So as you see, the forgetting is tightly connected to recall in a human memory.

And actually I also build this naive mechanisms to look in that if we collect the statistic of the retrieval and recall, and if it’s more often then probably it’s something that we wants to put some importance to it. And when we try to form this recall loop with the forgetting, then I proposed the different concepts of attention.

Focus, Attention, and Task Orientation

So we just remember what’s matter and we remember what’s matter to us in the particular action and task. And actually human memory is quite task-oriented and you quite often recall the things around the actions and by the way, they activate completely different areas of your brain. So you try to recall the information with the purpose to do something. So and your skills is slightly different kind of memory, it’s not declarative memory, and even if you have amnesia, you still know how to drive the bicycle. It’s yet another evolutionary thing, it’s more automatic and it’s quite interesting topic.

But then I guess that agent memory could be heavily built around the concept of the focus and attention. And then the forgetting is something that doesn’t matter right now for this particular task. And it’s quite task-oriented.

Explanability and Goals

And if it’s task-oriented, then we shift from the conversation and conversational relevance to more important thing that the context graph nowadays could solve: how you taking the decision, how to make this decision explainable, and how to make this decision of the agent following your goals, following your constraints, following your company policy and all other thing.

And I would say that this kind of context graph and agent memory architecture in a context of the tasks have completely different focus and attention mechanisms from the conversational memory because in conversational memory, you have only one cognitive process that are quite complex, or maybe couple of cognitive processes of digesting and extracting the memories and actually finding the relevant memories according to the conversational context. So it’s simple.

Social Dimension of Memory

But when we have the agent that actually communicate with other agents and actually cooperate with them and do something, so it’s already social component of the memory that agents needs to remember: who is reliable, to whom the agent could trust, to whom agent could delegate. So we get this social dimension of the memory for the agents.

And the most important that agent interact also with the humans that give him the tasks, and maybe some other agents give him the tasks. And it’s open question: should agent differentiate the human and the agent or for agent it’s all the agents and it doesn’t matter if this agent drinks tea while talking or just consume a bit of energy? So it’s a good question that we still needs to get answer.

Operational Memory and Symbolic AI

But the agent operational memory is more focus on the tasks. It’s more focus on the rules and what could happen and what’s to do if something happens. And then it’s open us the good old space of the symbolic AI and expert system and rule-based memory and rule-based expert system and approaches that we all forget because we heavily failed in eighties with the Prolog formal system and first-order logic and all other things. But right now all this stuff is more and more relevant.

Future Directions: Promise Theory

And what’s more important that rules relevant, we needs to have the memory to remember the rules, we needs to have the reasoning capabilities and solvers that could apply the rules and what’s more important, we just needs to have some framework that actually apply all this rule-based systems to the agent interactions and agent intents and all other thing.

And I touch this topic in my new book, What Goes Beyond the Context Graphs, and today we have the memory for sure that we already have a chance to talk. And we have the concept of the promises and Promise Theory of Mark Burgess. And in Promise Theory, we form some kind of promise graphs. And these promise graphs actually captures not just the decisions but some interaction of the agent with other agents, some interactions of the agent with the tools and data information.

So we have some promises of the interaction of the agent with the memory, we have some promises that interact with the tools. So somehow we have quite complex audit log that could be reflected, could capture what agents made, what other agents made for the agent, what kind of data traces we have, and all this complex promises could lead us with the casual analysis to the final decisions, give us the possibility to apply the rules and have some kind of explainability and causality.

Conclusion

So the combination of causality analysis, data traces, and actually the Promise Theory give us the good foundational framework to go beyond the context graphs because the context graph right now quite weak concept that say that we needs to store the decisions but we have no implementation of this. And actually I also talking about the semantic space-time that could be used for this.

So please, I have the bundle for the memory that explain all the things in two books, one is time-focused, another one I work on this beyond concepts. So if you’re keen to learn about the memory more, if you’re keen to build the agents that remember and do what you ask them to do, just read my books, contact with me, and let’s talk together about this complex topic.

To read my memory book bundle : https://leanpub.com/b/agenticmemory