重构AI时代的上下文引擎:如何让AI Agent理解组织背景与隐性知识
人机角色的演变与AI Agent的“入职之痛”
在现代软件工程中,知识的流动和积累正在发生深刻的变革。过去,人类开发者是组织中唯一的隐性知识(Tribal Knowledge: 团队或组织内部未被规范记录、仅在成员口头或经验中流传的隐性知识与常规习俗)载体。我们需要在庞杂的代码库、设计文档、Slack讨论中穿梭,通过不断试错来建立起对系统架构的认知。
然而,AI智能体(AI Agents)的引入重塑了人机协作的边界。AI智能体拥有极高的开发效率,但它们也面临着独特的“新员工困境”:每次启动新任务时,其上下文记忆都会重置。这就像是一位刚入职的技术专家,在处理每一项任务时,都必须重新探索你的代码库、测试环境以及部署规范。
在AI成熟度曲线(AI Maturity Curve)上,行业已从早期的代码自动补全(以GPT-3.5 and 早期GitHub Copilot为代表)发展到Cursor等智能编辑工具。如今,大多数团队正处于第四到第五阶段,即尝试通过模型上下文协议(Model Context Protocol: 允许 AI 智能体安全连接和导航各类数据源的开放标准协议)和自定义技能为智能体导航。而未来,我们将走向高度自动化的“软件工厂”。如果缺少持续的上下文引擎(Context Engine: 一种为人类开发者与 AI 代理提供组织级上下文和隐性知识的系统)来输送组织背景,智能体将在庞杂的未知世界(Unknown Unknowns)中迷失,无法独立运行。
Original English Source
What we do at Unblocks is we build a context engine. I just want to do a quick sound check at the back to make sure everyone can hear me fine. Can you guys Yeah, we're good. Awesome. So at a high level, a context engine delivers organizational context to both your human workers and now increasingly your agents. Okay. So why why is that important? Before we go too deep on the mechanics of how a context engine works, I just want to talk briefly about the problem. So, what we're going to do is we're going to hop into our time machines and we're going to travel back to the before times uh before agents and uh discuss a little bit about what we used to do as humans uh before agents came into the picture. And so for years um you were the context layer. You um had to go and do things like this. you had to find things you were looking for, trolled all over different data sources, different discussions taking place. Um, and then through the codebase of course to try to build up tribal knowledge and, uh, throughout time as your code base progressed, um, you'd be, you know, fighting incidents and things like that. And your organization over time builds up battle scars um from all these all these different things building code uh documenting architecture and and dealing with outages and things like that. But now um we have a new problem because uh as we introduce agents to the picture um they suffer from all of these challenges except for one thing. Agents are like new employees. they reset their knowledge every time you start a new task. Okay? And so you can think of an agent like an expert software engineer um who's a new employee on boarding for the first time. Every time they have to rediscover your code base, how your organization builds tests um and how they deploy software with each and every task. Uh can I just uh put a put a show of hands for everyone that's seen this slide before by Vim? So this is kind of like u this is a good way to view where people are on what we call like the AI maturity curve. Um starting at the the far left uh this is kind of representative of autocomplete back in the GBT35 days. You know remember co-pilot and things like that. Um and then you know kind of move on to using cursor. Um and then from there you're you're think you're talking about how you can start to solve the context problem. So some people are building organizational wikis. Just smile if if this is kind of um bringing up memories for you. Um and then you know all these things are great except that uh how do you give agents access to this and what are the compounding problems that the scaling problems as you move forward well if you give MCP and skills to your agents um to teach them how to navigate and build context and that's kind of where uh people are today most people they're at the sort of stage four to five level okay and uh they understand that context is the bottleneck and they're trying to build solutions to solve it for their engineering teams. So looking ahead uh to all the way to eight with software factories. This is kind of where the puck is going. I'm not sure if if folks were at the keynote this morning, but um it's it's all about like delivery of context and unknown and unknowns. And this becomes increasingly important as people start thinking about full automation of agents. they just can't operate without organizational context. They get lost. So, you know, like that's the real problem.
海量上下文的陷阱与“检索满足感”认知偏差
面对AI智能体缺乏上下文的痛点,业界最直观的做法是将整个代码库和架构文档倾倒进长上下文窗口。然而,这种“暴力注入”在实际研发中并不可行。一方面,百万Token的窗口依然无法容纳庞大的组织历史;另一方面,无关信息的涌起会严重分散AI的注意力,浪费Token并增加耗时。
此外,获取信息(Access to information)并不等于理解信息(Understanding)。如果仅仅将维基文档或代码仓连接到智能体,它们很容易陷入认知偏差——即医学放射学中的检索满足感(Satisfaction of Search: 医疗放射学领域的术语,指查找到第一个可能的病灶后便停止检索,在 AI 检索中表现为找到首个看似正确的答案即终止探索)效应。AI智能体在检索到第一个看似合理的代码段或配置后就会停止搜寻,从而遗漏更深层的潜在风险或依赖关系。
在软件资产中,代码只是露在冰山之上的显性呈现,而开发者的设计意图、团队特有的开发规范、Slack里的历史决策以及架构演变背后的权衡,全都隐藏在冰山之下。缺少上下文引擎对这些隐性资产的提炼与沉淀,AI智能体就无法建立全局视角。
Original English Source
Access to information doesn't equal understanding. Um I I know that folks are probably familiar with claude.md um and and uh and wiki layouts and all these things. If you attach a wiki, it still doesn't tell the agent where the information is that it needs. It can search for things in the wiki, but then what happens is it'll suffer from something that uh radiologists uh call satisfaction of search. So, this is a term in radiology where you look at an X-ray and you're trying to find a region um that might be an indicator for cancer. Okay? And you discover like one indicator and if you stop there uh you might miss other important indicators that might you know lead to diagnosis of even more uh issues. So this is what happens with agents. They don't they they they find something that they they think is correct and then they stop. Um the the other thing about agents is that they don't distill understanding. They can look around, they can find information, but they they don't understand how all the pieces fit together because without doing that leg work ahead of time. Um, they don't understand how, you know, your dependencies interact with each other and how your architecture and sort of future planning is going to scope the work that it does next. And so some some people will then ask, well, what if we just take the entire codebase and all of our architecture documents and just slam it into the context window. Um, and then yes, maybe like your agents will reason about everything all at once. And in practice, that that of course doesn't work. Um, not just because you've got way more organizational context than can fit into a context window, even one that's a million tokens in size. Um, but it it it causes the agent to get distracted. When you're working on a task, you want task specific flow. Um, and so your agents will get distracted easily if you give them things that cause them to look this way in that way. Um, and it'll just waste tokens and time. So, in this morning's keynote, um, Tariq from Claude Code mentioned unknown unknowns. I just want to uh harp on that phrase again. And it can be phrased a different way, which is finding the things that really matter. And so this is what your agent can see at the top of the iceberg. They can see the code and they can operate on the code. What they don't see are things like the actual intent, the team conventions, past decisions, things that you've discussed in Slack, for example, uh architecture rationale, and so on. And that's why your agents need a context engine to get real work done.
Unblocked上下文引擎的多场景实操与效能增益
为了打破这一认知瓶颈,Peter Werry在演讲中现场演示了Unblocked上下文引擎的核心功能。
对于人类工程师而言,当询问系统内部组件源码标记引擎(Source Mark Engine)的架构时,Unblocked不仅能基于代码的当前运行逻辑和未来的规划方案自动绘制出架构图,还能清晰展示推导线索(Show your work),帮助团队校验并持续修正知识库,构建坚实的人机信任。同时,Unblocked能静默监听Slack等沟通渠道,在置信度较高时主动提供解答,或直接作为交互式助理提供即时反馈。
在AI Agent的开发实战中,上下文引擎展示了其强大的降本增效价值。以优化源码标记计算器(Source Mark Calculator)为例:
- 在没有Unblocked支持的情况下,AI Agent只能通过漫无目的的代码库检索来摸索算法。虽然最终也能制定出优化方案,但耗时达2分钟,并且极易因前提假设错误而陷入往复修改的闭环。
- 在Unblocked的支持下,系统会自动为其精准输送此前关于该组件的PR讨论、Slack对话以及相关的架构备忘录。智能体仅需1分钟便制定出了直观且切中要害的优化方案,Token开销大幅降低(控制在1美元以内)。
这种效能提升在长线任务中具有显著的复利效应(Compounding effect),能够有效切断AI由于错误前提导致的死循环迭代。
Original English Source
So, I'm going to now attempt a live demo. And hopefully the demo gods are kind. Um, so I want to pop back up conceptually. Oops, I think I'm on the wrong tab. We'll get to that one in a sec. So for now, sorry about that. And here we are. So I'm going to ask a question as if I'm a, you know, I'm a human and I want to get some information about my codebase. And, you know, the human layer hasn't gone away. We talk about agents and their need for context, but um humans are still asking questions about the codebase and we need that level of understanding because ultimately the accountability stops with us. When you hit merge on a PR, you need to understand what it's doing um and you need to understand how the architecture works. So this question I asked here um is about an internal component of our system called the source mark engine and you can see that it uh is able to articulate it fairly well. um understands the architecture. This this diagram here is uh is generated. So it this diagram doesn't exist. Um it just figures it out based on the um the way the code operates today and then some proposals for future architecture. And then uh what's really important is that you show your work. This is a trust building thing more than anything, but it allows people to see if um if the answer is maybe not entirely correct, then you can in look into the uh the knowledge base that you have and make corrections. Increasingly agents are doing this for you. So now um what I want to show you is another place where humans spend their time which is in Slack and this is where a lot of the decisions get made of course. So I can do something like this. And uh unblocked will sit and kind of listen for things that are things that can chime in on when it provides a high degree of Oh, sorry. We went to the wrong You guys can't see that. Thank you, Claire. Oh, come on down. Let's see if I can bring it up. There we go. Perfect. So I can ask questions like this in unblocked and if it thinks it can chime in on the answer then it will chime in. Otherwise I can just um address unblocked directly and ask the same question and when it thinks that it has an answer to give then it will give an answer and so we can get um quite a bit of interesting content there from unblocked. Thank you. Unblocked. I'm going to switch up and show you the the really interesting thing which is the agents. Okay. So, um in in that question, the source mark engine, I'm not sure if people picked up, but there was a little thing at the bottom there that said, you know, there's some optimization opportunities. Um so what I did here is I went into claw code and I asked it um without using unblocked to um uh generate uh a plan to optimize the source mark calculator and it did that and it happily went and you know searched through the code and and tried to figure out how the algorithm works and so on. Um and it it reached a conclusion that's great you know it does a pretty good job um but you know it maybe could do a little bit better. So, I asked that question again uh using unblock this time and it it really kind of nails the the nuances because it picks up on the the uh PRs that we um where we discussed future possibilities for improvement. um some Slack conversations that we had and uh of course you know notion and architecture documents and it shows its work and this is really important because um all of these things here the sources come back to Claude and then Claude knows exactly where to jump to next if it needs to elaborate on that context. And so I just want to show you what the impact of that is. So if I um Whoops. Thank you. If I pull up usage here, you can see that with unblocked, uh, the total cost was, you know, subd dollar to create the plan. Uh, took about a minute. Ignore the wall clock time because I've had this open for about an hour. But, um, it's about a minute. And then if I look at um the usage without unblocked, you can see that it's about 2 minutes. And and and it costs more to generate all that context. Now, the reason that happens is because it has to do more work. It has to look around. has to discover things. Um, and this compounds, not only does it have to do more work to discover things, it doesn't discover the right things. So, when you get further down in your execution, it may be operating on the wrong plan or the wrong assumptions. And then you have to go back and you have to loop over and over again. So, the real value of a context engine is not like the upfront cost on these short tasks. It's the compounding effect. Um the the other Tariq from Sonar mentioned this in the keynote this morning and it's true like the loops compound and you have to be like efficient the entire way through with your context.
代码库外的隐性知识:社交图谱与开发最佳实践
更深层次的组织上下文不仅包括静态的代码和文档,还包括由开发人员组成的动态协作网络。Unblocked能够分析历史拉取请求(PR)等数据源,自动提取并沉淀代码库的开发最佳实践(Best Practices: 在特定环境或代码库中被公认最高效、最安全的编程与架构规范)。
这些最佳实践可以无缝融入代码评审流程。在演示中,当代码评审智能体运行并指出问题时,Unblocked能自动关联并呈现团队资深工程师 Richie 在历史类似PR中留下的关键评审意见。通过将资深工程师的经验作为高权重信号(Expertise signal)注入,AI智能体能够更准确地对代码进行审查。
此外,Richie曾通过Unblocked成功排查了评审问题报告率暴跌的问题,指出其根源在于团队将底层模型切换为Claude 3.5 Sonnet(转写中记为 Claude 48)后引入的智能体行为差异,并自动关联了相关的Slack讨论,一键生成了修复PR。
为了推动上下文引擎生态的发展,Unblocked开源了两款分析工具:文档查询引擎(Document Query Engine),允许开发者通过解析历史PR自动合成schema并提取项目洞察;工程社交图谱(Engineering Social Graph),通过展示团队的Review协作关系和代码库的专家覆盖盲区,量化团队的研发健康度。同时,官方提供的上下文引擎模拟器(Context Engine Simulator)也让开发者能够以零门槛的方式,直观对比有无上下文支撑对智能体开发效率所带来的颠覆性改变。
Original English Source
I'm just going to jump back to Safari and I'm going to point out um some really interesting things. So we also have a a code review agent. And when we say um you know organizational context, we're talking about more than just the underlying data. Uh we're talking about real intelligence. So what unblock does is it looks at um not like it looks at pull request data and there are other data sources for this and it generates a series of best practices that help align agents to your codebase. But we thought that this would be really helpful to surface for the review agent as well. So what you can see here is um it unblock chimed in and then Richie here said, "Oh, that's cool. That's something I would say." And that's because that actually was something he said. So it surfaced the uh the previous comments. Richie's one of the senior engineers and we use the sort of seniority or expertise as a signal um to boost uh comments that are important. Okay. So another uh interesting interaction by Richie, he uh discovered that the number of code review issues that were being surfaced dropped uh precipitously and he was debugging it with unblocked. Um he got all the way to the bottom and realized what roughly what the problem was and then asked unblocked to fix it. Now this this is something that we have internally um you know that we're experimenting with. Um, so unblocked uh can run as an agent in the cloud. Um, but what's really cool about this is that it has all your organizational context at its fingertips and the results are are pretty magical. So it can do things like generate this PR um, and then what you'll see here is that not only does it generate the fix, it also is able to relate it to the all the conversations that were happening. So this PR was created because and you read that context thing. It's mind-blowing. After this PR, we switched to uh Claude 48 and it dropped a ton in issues because of the behavior is quite a bit different. So then it said Richie directly correlated the drop. Now what's this thing here? Let's click on it. It is a Slack conversation. So, it found the Slack conversation, correlated all of that, you know, past history back again, and then we ended up with a with a final PR. So, um, I'm going to I've got only a few minutes left. I'm just going to close this out really quickly. We have a uh a couple of open- source projects that are kind of interesting if people want to play with them. One is the document query engine. That was, uh, something that I talked about on Monday in my workshop. Um, I may uh talk about it again tomorrow, but I just want to give folks a sense of what this thing does. Um, whoops. If you want to play with it, it's open source, so you can just download it and have it go. It basically runs over your um uh GitHub repository, ingests uh your your historical pull requests, and then uh synthesizes a schema based on the documents that it can sample. Um and then from there you can issue any kind of queries that you like and get all kinds of insights out of it through the agent chat. You can ask all kinds of questions. Um and then lastly the engineering social graph. So this is the thing that I was talking about earlier that helps us pin down expertise and team relationships. Um so what you can see here is this sort of like the rough breakdown of our team structure at Unblocked. As you can see we're a fairly small team. Um and so we've got these um uh these clusters of people and how they relate to each other indicates the kind of um review relationships that they have. So these are you know these lines show like we review each other's code. Um we can then cluster that and generate team labels for that or show the coverage across your codebase. This is really cool. you can see kind of where the holes are, where you might be lacking expert coverage. Um, and that's exactly what we use within the context engine itself. All right, one last thing we have uh for those that want a taste of what a context engine can do but don't want to sign up for unblocked right away. Um you can use uh something that we call the context engine simulator which will basically build up a context behind the scenes on a per task basis and then use that context uh to to drive the task. It'll do it with context and without context so that you can see what the differences might be. This is a QR code for that if you want to just take a quick snap. Awesome. And I'll just land on a quote from one of our customers. 50% fewer tokens, faster triage, better answers. And that's exactly what a context engine can do. One last shout out um before we end. My colleague Brandon is giving a talk in 10 minutes uh at room 2020. um he's going to speak to in a lot more detail about some of the higher level things that context engines can do. I'm going to run over there right after this and I think all of you should follow me. Awesome. Oh, and don't forget to get a coconut. >> [music]