为什么更大的上下文窗口拯救不了你的 Agent：上下文工程与记忆架构实战

Greetings. First, I must say, I am Elizabeth, not Morgan, but on the agenda they never changed it. So we are almost the same, only hair color is different, okay? So everything will be similar. Today I will tell you about why infinite context window is a misconception. Yes, this is because right now agent systems break when tools return large amounts of data. The context window overflows, the agent starts hallucinating, and its reasoning capabilities get worse and worse.

So now context engineering helps us build a better future for these agentic architectures. We have some methods that allow improving the context in the window for these agent applications. Imagine you have an agent monitoring a program, and this agent must extract logs from that project. Every time the agent receives those logs, it adds that data into the context window. Each time, this context window is growing and growing continuously, because in a session every request pulls in the previous ones since the agent has to remember everything.

Earlier, about a year ago before the agent era, we thought more tokens would fix everything. They said a bigger context window would solve the problem. But today we know that is not true because of the model's attention curve. What is this? We have a large amount of data, but if there is too much, the engine forgets the middle position of this curve. It only remembers the beginning and the end of what we sent in the context window.

Context engineering is about providing a model with the necessary information exactly when it needs it. Sometimes it is unnecessary for the model to have all information included in the context window. Sometimes an overflowing context window hurts the model because it can accept prompt injections. Context engineering helps optimize the window for best background results and saves costs.

Today I will show you several methods that will help improve the use of context windows. I merged these methods into strategies: First, we use external storage to move big data to permanent storage, and you can create memory pointers to store this data. Second is selection: you pull out only what is relevant, squeeze and reduce tokens by summarizing, and return that info. And the last is isolation: you can separate context between agents.

Let us start with the conversation manager strategy. We have many strategies, but let us start simple inside a framework. Here we will use the agent framework Strands. It is an open-source framework we support, free and completely model-agnostic.

Inside Strands, you can use three different strategies:

1. Sliding window, which allows storing only the latest information.
2. Summarization inside the framework, allowing you to generalize older information while keeping the last few messages.
3. A combination of the previous two, merging the summary of older things with the last few messages.

With Strands, you only need one line of code to create a working agent loop. That line creates the entire workflow behind the scenes. You do not need to create nodes or count tokens manually, because Strands does that for you with system prompts and tools. It uses Amazon Bedrock under the hood, but if you want to use Ollama, Gemini, or OpenAI, you just add one line to import and switch the model.

Returning to our conversation manager strategy: you can specify a summarization ratio, such as 50%, and specify that you want to preserve the last four messages. Every time you invoke this agent in the session, it will maintain your context window this way: preserving the latest four messages and summarizing the remaining 50%.

What should you do when you have a lot of memory and you do not want to lose everything in the context window? You can use short-term memory, because it preserves recent conversational history in the current session. You can use the summarization and sliding functions I just mentioned for short-term memory.

But what happens when you start a new session with the agent and do not have that short-term memory? You can have long-term memory and create a vector database for it. You can have different types of vector databases and retrieval prompts that understand your conversation and save everything you need.

And the third type is relational memory. You can use graph entities—you can look into Neo4j manuals for more details. You can perform graph traversals and link connections.

As you can see, you can use a combination of all three:

• Use short-term memory for the current session.
• Access long-term memory for past historical records.
• Add graph memory if you have entity connections you want to preserve.

What should you do if your project generates huge volumes of logs? You don't want to put all these raw logs into the context window every time you call the agent. You can save this using indexed memory and memory pointers.

How does it work? You have a tool that returns a bunch of logs. You use that tool to fetch logs, save them into external storage, and return an identifier for that entry. Using the agent's state condition, you store that identifier as a memory pointer in the context window. Then you can use another tool that can understand those logs by passing messages like: "Hey, what happened in the logs for this identifier?" You don't need the agent to see all raw records to understand the situation.

If inspecting everything is strictly necessary, you pass the identifier: "Here is my identifier, please inspect these logs because I need to understand them." In code, inside the agent, you build this architecture using tool decorators. You receive program logs, capture the info, save it, and create an identifier memory pointer stored in the context window.

If your application is too complex and one tool is not enough, you can use a multi-agent swarm architecture. You create multiple agent instances and split their context windows. You isolate contexts so one agent's garbage doesn't pollute another, because agents don't need all shared information between them. You share state via call state pointers and identifiers across the agent swarm.

Next: what happens when an agent does not want to stop? Sometimes an agent gets trapped in an infinite loop, invoking the same tool again and again, never ending. You can prevent this by placing hard limits on loop calls. When a tool doesn't provide a clear answer, the agent keeps calling. You avoid this by providing explicit answers from tools and limiting the maximum number of tool calls—for instance, allowing a flight search tool to be called at most three times. Once limited, it will never execute that inner infinite loop.

Lastly, how to handle external integrations: sometimes we call MCP tools or external APIs with long response times. In AWS API Gateway, there is a maximum timeout waiting for a response. If your program doesn't respond within that time, it halts with latency errors. You can avoid this using asynchronous methods.

In synchronous calls, your runtime waits forever for the MCP response. With asynchronous handlers, you call your external API/MCP, save the task status, and retrieve the result during a subsequent call. For example, using FastAPI you create two tools: one to start the long-running task, and another to verify task status. This asynchronous pattern makes a huge difference; otherwise, while waiting, the agent stops responding and cannot fetch information.

So, what should you avoid?

• Avoid context overflow. Do not include everything there. It is not worth adding everything to the context window because the agent does not need all that information. Only provide what the agent needs to create the response.
• Be careful with built-in summarization: sometimes when summarizing, the agent might forget necessary details. If you summarize blindly, you might lose things required for the agent's answer.
• Avoid passing raw, uncurated data to the agent in full sequence.

What are the best practices?

• When a tool returns heavy data volumes (large database records, extensive logs), offload it externally and use memory pointers.
• For multiple agents requiring shared heavy data, coordinate state across steps using identifiers.
• Keep agent cycles and tools explicit: ensure tools return distinct, clear answers, enforce tool call limits, and implement asynchronous descriptors.

Thank you. If you want to know more about agents, we have a booth with specialists and AWS Lego mini-blocks. And remember, I am Elizabeth, not Morgan. Thank you very much!