将智能体记忆转化为可执行技能：基于 Neo4j 上下文图谱的架构演进

Let's get started. I see that some people want to join, so please come in. This session will be dedicated to actionable knowledge and context graphs. So, my name is Will. I am a product manager at Neo4j. Em. There's been a lot of talk about cycles at this conference, hasn't there? React loops, ReAct loops. Sometimes I feel like we're in some kind of amnesia cycle, right? When our agents go through the reasoning phase, they act, successfully complete the task, but then they seem to forget what they learned. Today's memory systems are mostly focused on embedding text, and during retrieval, on finding the most relevant data to insert into context, right? Um, something like this. We find similar pieces of data in our corpus, insert them into a context window, and hope that the agent can do something useful with it, right? The problem is that remembering is not exactly actionable knowledge. One of the challenges we face is, for example, the lack of a canonical representation of an object. If we have three different ways to address Dr. Nguyen: Robert N., cardiologist, correct? Depending on the context of the discussion, we need to have some canonical representation of the object. In fact, agents need knowledge that is coherent, typed, and traceable. Not just ones that can be obtained. Search is only part of the problem when we talk about agent memory. So, in Neo4j, we see agent memory like this. We view it as a connected graph consisting of short-term, long-term, and logical memory. We'll look at this in a little more detail. But be patient with this idea of context graphs for agent memory, okay? What are the main components here? Well, first, it's a transition from unstructured data to a knowledge graph. Moving from the process... agent messages, right? Both user and assistant messages go through an entity extraction process where we determine what these entities are and how they are related. Doing this in a graph where we have clear types, right? We have a relationship that describes how these entities interact with each other. The entity recognition phase is one of the most important parts of building this knowledge graph, right? Understanding and ensuring that you have a canonical representation of an object. So when we talk about Dr. Nguyen, we know who the provider is, his name, his role, etc. And a shared ontology is an important part of that, right? That is, having a certain description of your data model, the domain you are working with. This is one of the key elements that ensures a successful transition from unstructured text data to a knowledge graph. So for those who have worked with agent memory before, this should look a little familiar.

One thing that I think is really important when we talk about agent memory and real knowledge is the idea of a reasoning graph, right? If we've seen our three components of agent memory, reasoning memory is the primary one here, right? We talked about short-term and long-term memory. These are the messages, the entities extracted, but what about the actual actions the agent performs? What about the reasoning process itself? We want to make sure we capture that because it's an important part as well. So, we want to make sure we're preserving the thinking and the facts, not just the data itself, right? That is, how exactly the agent made the decision. And that's usually represented as a decision-making trail, right? So every decision our agent makes involves evidence-based reasoning, right? We have policies that we model explicitly, and we understand the execution plan: the tools that the agent called, and the results of those calls. We also want to record things like the number of tokens spent, execution time, etc. This is all part of fixing the memory of reasoning. And this is important not only so that one agent can perform better the next time they do the same task, but also for systems where you have hundreds or thousands of agents sharing a common set of tools, right? So we can store these agent runs, these decision traces in a graph, and then share them with other agents in our organization, right? Thus, we have one shared context graph that allows our agents to learn from each other through this shared memory element. Okay, that's the thing about memory. This is, in fact, the system we built in Neo4j. A template that many follow for working with agent memory.

But what about the idea of practical knowledge, huh? How do we allow our agents to act? Memory, one way of thinking about memory is that it's a beautiful reflection of what happened, right? We know the people we were talking about and how they are connected. We have these traces of decisions. The next step is usually skill building, right? How many people use skills with their agents today? Most of them are cool. How many people wrote these skills themselves? Cool, most of them. How many people have asked an agent to write a skill for them? Steeply. So, that's what we're talking about here: how to move from this memory graph, this context graph, to how to use it to create grounded and actionable skills. So, if you're not familiar with the concept of a skill, it's an open standard. Anthropic released it. Agentskills.io seems to be where this open standard is hosted. The main idea here is that we have some metadata, right? Some description of what this skill is about. And then it's a gradual disclosure of information, right? That is, I have a lot more detailed information. These are often files in markdown format, links that we can gradually select, if we go through the process of performing part of the skill, we can retrieve and load this data into the context. So this gives us common units that we can take, package the skill, and reuse across different agents. But skills have a similar problem to what we saw earlier when we talked about memory. Namely, that we usually work with prose. And so we're still limited in understanding whether this is a canonical object and in providing debuggable steps and results. So, to solve these skills problems, some of the Neo4j research team did some interesting research and published a paper on APE—a graphical representation for training and managing agent skills. This is a screenshot from the article. Zach is here somewhere. If it's not here, it's at the Neo4j booth today, so be sure to chat with Zach if you're interested, but you can think of APE as an extension of the agent skills protocol with additional metadata, which now treats your skills as typed execution graphs, right? So we have steps modeled as nodes, and we have a very strict scheme that governs the description of how these steps become actionable. Yes? So, think of it as a way of representing a skill as a graph, broken down into steps that are performed using a schema that governs the description. This is the APE protocol, and according to the benchmark used in the article, it really makes a difference: when applied to Skills Bench, we saw a significant increase in the number of tasks completed when using the APE protocol for human-selected skills.

So, this looks interesting. How can we use some of these ideas and research to distill skills? Essentially, we want to take this memory graph, this context graph, which may be limited to a workspace or a project in an organization, and distill it into a skill, but not just as a markdown file. We want it to be based on real data that we have observed. We want it to be deterministic, right? We want to have well-understood procedural steps in our graph and we want to be able to manage it over time, right? If the underlying data that shapes our skill changes in memory, we want to be able to understand it and know about it. Justification is the important part here, right? Making sure that the data that makes up our skill is based on our memory system, but that's not enough for it to be useful, right? There are other heuristics that we should pay attention to. For example, coverage: do we make sure that the steps and descriptions throughout our skill are valid within the entire skill. Coherence. Coherence is interesting. This is a way we can detect whether the information or subgraph coming into the skill distillation is divided into multiple topics, and we can suggest: maybe you should create multiple skills here, and so on. So, these are some of the elements that lead to skill generation. Skill management is an important aspect that I mentioned, making sure that we can understand: as the skill changes, as the data changes, are we able to update that skill? And thanks to dynamic loading, you can think of it as a managed skills inventory, which allows us to get the most relevant skills for any agent and understand whether they are outdated or have lost their relevance, right? Steeply.

So, we implemented this in the Neo4j Agent Memory Service, or as we call it, NAMS. This is a memory service for agents that is part of our development at Neo4j Labs. We also have open source tools that implement these patterns. Essentially, it works like this: we have a context graph within the workspace, right? It contains three types of agent memory: short-term, long-term, and reasoning memory. We have background processes capable of performing this distillation process. We'll see what that looks like in a minute. We also have a background creation process that provides control to detect when our skills become obsolete based on the data contained in the memory system. This is the pipeline we use to create them. I won't go into detail on this. The only thing we want to point out here is that most of these steps are deterministic. We're really only using LLM here to synthesize some statements and generate some of the text that we use to describe skills. And one more thing I want to point out: the first stage is scoping. That is, where do you start distilling one of these skills? Is it related to a specific entity, or to the subgraph around it? Is this a specific conversation? Well, that's another thing. So we can decompose the way the skill is represented and make sure that each of these components is again based on the original data. This is exactly the point where we check if the skill becomes obsolete if any of this source data changes, is removed, or becomes inconsistent. Steeply. And then we can also, as I mentioned before, combine skills, right? This is where the principle of consistency comes in handy. We use graph algorithms like community detection, right? So if we are mixing topics from multiple communities, it could be a sign that we need to decompose our skill, and we will find this within skill management. Steeply. So, I have a few minutes left. Let's see what it looks like. So this is NAMS, the Neural Agent Memory Service. It is currently free. Anyone can log in and try it out. This is what the dashboard looks like. The basic idea is that we have a REST API and MCP tools that we can provide to our agents to write, retrieve, and work with long-term, short-term, and logical memory. We go through the process of entity extraction and recognition, so I can view a graphical representation of my agent's memory. I can navigate it, etc. We can see the entities that have been marked. An important part here is the concept of ontology. In this case, we use a medical ontology. We will work with data on visits, medical professionals, etc. I also uploaded a few conversations here. We can see the conversations that were uploaded regarding the medical agent, right? Now we are ready to highlight the skill. As we said, the first step is to determine the scope of this skill. We can do this for the entire workspace. Although this is not always what we need. We can do this for a specific entity, specific conversations, or a class in an ontology. Let's do this for our last conversation. We will see how the process begins in the distillation queue: the system will receive data, go through a seven-stage pipeline, form a skill in the form of a graph, and package it for us. While it works, here is one example. Let's just take a look at him. Here is the example we ran earlier. It was launched based on a conversation during a patient's appointment. As you can see, we have essentially highlighted the steps that make up a skill: from receiving a patient to filling out their medical record. We see that each of these steps is based on the actual tool calls and entities that form the core components of the skill, and we package this together with the skill file. MD. If we were to upload this, it would be packaged with other reference materials in accordance with the progressive disclosure standard we use for agent skills. Class. So, this was a quick overview of how we look at agent memory within this context graph with Neo4j, and I'll leave you with some useful links. The slides can be downloaded here. Here is a link to the slides and a QR code. The Neo4j agent memory service I mentioned is also listed here, along with a bunch of documentation and resources for our open source tools. So, that's all. I'm out of time, but we have a Neo4j booth. So I will be there, as will many others from the Neo4j team. See you there. Thank you all.