独立 AI Agent 构建者的宿命:终将重塑简陋版的 “CI/CD” AI Engineer 2026-07-11

独立构建的工程代价:五种被重新发明的软件保障

当你开始独自构建 AI 智能体(Agent:具备自主规划、工具调用和执行任务能力的 AI 系统)时,很少有人会提前警告你一个事实:你起初以为自己只是在调整提示词,但如果你构建得足够久(特别是独自构建),你最终会发现自己正在以一种非常痛苦的方式——即在一次又一次的系统失败中——从零开始重建一套类似于持续集成与持续部署(CI/CD:Continuous Integration and Continuous Delivery,通过自动化步骤保障软件高质量交付的流程)的体系。

Sumaiya 运行着一个包含 19 个技能的云端代码智能体系统,涵盖写作、研究、知识库同步、分析同步、钩子处理及字幕处理等。在这个复杂的系统中,最核心的收获并非如何写出更好的提示词,而是意识到自己在笨拙地重建五种控制机制。在典型的智能体内容管线中,信息需要在多个步骤之间传递(从调度器到命令、到研究、到内容规划、到生产、到校验、到评审,最后进入输出文件夹)。在这个链条中,每一次数据交接(Handoff:不同模块或智能体之间的数据传递环节)都可能成为系统产生欺骗的温床,而独自构建系统意味着只有你自己能发现这些问题。

当你在这种多步骤系统中修改了某个提示词或技能,下游就会发生崩溃。为了解决这个问题,你不得不构建一种验证输出是否符合预期结构的方法,这实际上是在重新发明回归测试(Regression Testing:验证代码修改未引入新错误的测试方法)。接着,你设置了定时任务,某天它默默失败了而你一周都没发现,于是你开始增加警报,这又是在重新发明持续集成监控(CI Monitoring:对自动化集成状态的实时监控)。当某个技能的输出结构发生改变导致下游三个技能同时损坏时,你在边界处增加校验,这相当于重新发明了契约测试(Contract Testing:验证不同服务间交互接口是否符合契约的测试)。为了防止未就绪的成果被直接发布,你在就绪文件夹前设置了检查点,这正是重新发明预发布环境(Staging Environment:用于在最终发布前进行完整测试的模拟环境)。最后,当系统出错且难以定位具体是哪个提示词、技能或交接步骤产生异常时,你开始记录所有数据,这便是在重新发明审计追踪(Audit Trail:详细记录系统操作及状态演进的历史日志)。智能体系统默认并不提供这些保障,这逼得开发者不得不独立地用最糟糕的方式去重建它们。

Original English Source

Here's something nobody warns you about when you start building agents alone. You think you're building prompts. You think your building skills are worthless. If you build long enough, specially alone, you will start building something completely different. Something that looks suspiciously like CICD. Except worse, because you're building it from scratch, one failure at a time. I'm Sumaiya. I run a 19 skill cloud code agent system. Writing, research, vault sync, analytics sync, hook, transcript, and many more. And the most useful thing I learned from building this was not how to build better prompts. It was recognizing the five controls I was rebuilding badly and what you can do instead. Before I show you the problem, let me show you the system that taught me the problem. This is the agent content system. It is open source. Link in the description. It runs every other Saturday. It reads from a knowledge vault, creates a research brief, builds content plan, produces 12 content pieces, then runs verify passes, reviewer gates, deduplication, and finally saves the output as markdown files. But here's the thing that matters for this talk, not the content. What matters is that this system has seven handoffs. Scheduler to command, command to research, research to content content plan to production production skill to verifier, verifier to reviewer, reviewer to the output folder. Every single handoff is the place where the system can lie to you. And if you're building the system alone, nobody catches the lies except you, usually after the damage is done. Here's the pattern I want you to watch for in your own systems. If you build agents independently, you will rebuild these five things roughly in this order. You change a prompt or a skill, and something downstream breaks. So, you build a way to test whether the output still matches the expected shape. Congratulations, you have reinvented regression testing. So, you set up a cron job or scheduled task. One day, it silently fails, but you haven't noticed for a week. So, you build alerts. You just reinvented CI monitoring. One skill changes its output schema, so three skills downstream break. You decided to add a validation at the boundary because of it. You just reinvented contract testing. An artifact looks done, but it shouldn't ship. So, you add a checkpoint before it goes to the ready folder. You just reinvented staging environments. Something goes wrong, but you cannot find out which prompt, which skill, or which handoff had the bad output. So, you start logging everything. You just reinvented audit trails. The reason the title says worst version isn't because agents are software builds, it's because you end up needing the exact same operational guarantees. However, the agent systems give you none of it by default. So, you build them independently. The worst version. Without even realizing that you're building it.

智能体的隐性失败:警惕包装精美的“合格”废品

在智能体系统中,最危险的失效绝对不是那些一眼就能看出的“糟糕输出”,因为显性的错误非常容易修复。相反,真正具有破坏性的是一个在表面上看起来十分完美的产物——它拥有所有必须的章节、标记为就绪状态、文字通顺易读,因而成功通过了所有的格式验证,但它却存在致命缺陷。例如,它使用了完全错误的语气,或者包含未经证实的断言,又或者重复了陈旧的观点。然而,由于它符合格式规范,系统依然将其判定为合格并准备发布。这在软件工程中就相当于代码虽然编译通过了,但所有的单元测试实际上根本没有运行。

为了具体演示这一问题,Sumaiya 展示了一个精简版的智能体内容管线,包含了三个典型的“智能体谎言”及其捕获机制。

第一种失效是语气漂移(Voice Drift:智能体输出偏离了预设的个性化人设或特定语调)。例如,智能体输出了这样一句话:“释放 AI 采纳的力量。这个颠覆规则的技术将彻底变革当今快速发展企业格局中团队的运作方式。”这种句式充斥着领英(LinkedIn)上泛滥的 AI 营销黑话,它既不符合演讲者本人的表达习惯,也不符合任何真实人类的自然表达。但在无防护的放任模式下,系统仅仅因为该文档包含了所有要求的栏目且格式完整,就直接将其放行。而一旦开启了守卫模式,在语气契约(Voice Contract:用于检测生成文本是否符合预设语气和表达风格的验证规则)的拦截下,这篇语气漂移的文章就会在进入发布就绪文件夹之前被彻底阻断。这表明,校验门槛(Gate)的价值不在于直接提高内容质量,而在于赋予系统说“不”的能力。

Original English Source

The dangerous failure in an agent system is never a bad output. A bad output is very easy to fix. You glance at it, and immediately you can understand it's a bad output. The dangerous failure is a polished artifact that looks great at a glance. However, it will never pass your exit gates. It uses the wrong voice pattern. It makes an unverified claim. It repeats an old angle. It's missing required sections, and it gets leveled ready to publish anyway. That is the agent equivalent of shipping because the code compiled, but the tests never run. This is what I'm going to demo for you. Not the happy path, but the three ways the agent can lie to you and the gates to catch it. Let me start with the happy path because this is what all agent demos show you. I'm going to run a small privacy-safe version of my content engine pipeline. This isn't the full repo. It's a distilled version of the production problem. The pipeline is simple. Generate a content artifact, run gates, and either allow or block the output. Look at the output. The markdown has a caption, pinned comment, visual brief, verification log, vault assets, production notes, and a ready status. If I demo only this, the system looks done. The artifact looks professional. The content reads well. This is why agent demos are misleading. They always show you the happy path. But what happens if the path is not happy, but the output still looks ready? Now, let's look at the failure number one, voice drift. Look at the content. Unlock the power of AI adoption. This game-changer will transform how teams operate in today's fast-paced enterprise landscape. If you have spent enough time on LinkedIn, you have seen this exact same sentence thousands of times. It's generic AI marketing language. It's not my voice. It's not your voice. It's nobody's voice. But, knife mode saved it anyway. Because the artifact has all the required sections. It has a ready status, and it looks complete. Now, what what happens when I add one boundary? Guarded mode blocks it at the voice contract. The pipeline stops before this artifact enters the publish ready folder. This is the point. The gate doesn't make the content better. If you're building a content system, this is the first gate I would recommend. If you're building any other agent system, the equivalent question is, what does wrong voice looks like in your own domain?

事实性与独创性校验:如何利用防护门槛阻断劣质资产

第二种失效是缺失事实核查(Missing Verification)。在生成的文案中,智能体写道:“拥有清晰语义所有权模型的团队可以减少 37% 的 AI 落地返工率。”这个 37% 是一个非常具体且极具欺骗性的数据,但在检查底层的验证日志时,里面却是一片空白。这正是此类错误极其危险的地方:文字表达极具说服力,数据听起来也合情合理,但在没有任何事实核查和参考来源的情况下,它只是一个被精美包装的虚假结论。在放任模式下,这篇存在事实硬伤的文章顺利通关;而在守卫模式下,事实校验契约(Verification Contract:核对数据或观点是否拥有明确数据源支持的硬性规则)会立即将其拦截。如果智能体在涉及数据和用户反馈等重要领域发表了未经验证的言论,且缺乏追溯链条,系统交付的就只是缺乏可信度的内容。

第三种失效是重复切入点(Duplication Hook:智能体重复使用已有的选题角度或开篇钩子)。这是独立开发者最常遇到的痛点。智能体生成的内容在技术上是连贯且全新的,但它的开篇引入角度(例如:“当看板看起来没问题但工作流出错时,AI 落地就会失败”)与知识库历史记录中的某篇旧文几乎完全重合。这种重复利用相同选题角度的行为会让你的系统显得高度机械化,即使单篇文章质量尚可,读者也会比你更快察觉到这种套路感。在守卫模式下,系统通过去重校验契约(Deduplication Contract:对比历史库以确保选题及核心切入点具备独创性的控制机制)检测到了重复,将其拦截,并同时记入审计日志。

要构建一个在生产环境中真正可运行的智能体系统,关键不在于引入多么复杂的框架,而在于实施几个简单且枯燥的边界门槛:

  1. 输出结构契约(Pre-save Output Contract):在保存前,验证产物是否具备完备的字段和格式。
  2. 语气与业务契约(Voice/Domain Contract):检查内容是否符合专有的个性化语调和业务规范。
  3. 事实验证契约(Verification Contract):确保文中的所有关键主张都有据可查。
  4. 去重契约(Deduplication Check):核实选题或方法是否具有独创性,避免系统自我复制。
  5. 审计追踪(Audit Trail):在系统故障时,无需重写管线即可追溯具体出错环节。

在软件工程中,我们懂得不能仅因为代码能够编译就进行部署;在智能体系统中,我们也绝不能仅仅因为产物看起来完整就草率发布。解决问题的核心不在于消灭所有的智能体失败,因为失败是必然的,关键在于防止系统巧妙地“掩盖”失败并将其推向游。开发者应当首先梳理出从输入到输出的全部交接路径,并挑选出最昂贵的交接点(即一旦出错将造成最大损失的环节,例如公开发表错误数据或破坏下游接口契约),在此处部署你的第一个硬性拦截门槛。一个只会记录警告的门槛不是真正的门槛,它只是一个建议;真正的门槛必须拥有强行阻断流程的能力。

Original English Source

Failure number two is missing verification. This piece says, "Teams with a clear semantic ownership model reduce AI rollout rework by 37%." 37%. This is a very specific claim. Where did it come from? Check the verification log. It's empty. The prose is usable. The number sounds plausible. That's what makes the failure dangerous. A confident sounding claim without any verification or reference. And knife mode saved it. Guarded mode blocks it. Claim bearing content cannot ship without a verification trail. "Trust me" is not a verifier. If your agent system makes claims about data, about users, about anything, and you don't have a validation chain, you're shipping unverified assertions with a professional looking wrapper. That's not an agent problem. That's a credibility problem. Failure three, duplication hook. This is my favorite failure, because this is the most realistic one for solo builders. The output is new. The content looks technically coherent. But, the hook, the opening angle, is a near duplicate of something from your vault history. Yeah, adoption fails when the dashboards looks right, but the workflow is wrong. That angle has already been used. If your system keeps generating near duplicates, your system looks automated, even if every individual piece is technically fine. Your audience notices before you do. Garden mode blocks it at the data contract. I noticed this. It also wrote an audit record. That audit record is boring, but when a scheduled run fails at 2:00 a.m., the final artifact alone is not enough. You need to know which gate failed. Which contract was violated and why? That's the audit trail, the fifth reinvention. And that's the one most solid builders had lost after the damage has already been done. So, what's the pattern here? You don't need a platform. You don't need a framework. You don't need the ecosystem to catch up. What you need are a few boring gates. A pre-save output contract. Does the artifact have the required shape before it's saved? A voice or domain contract. Does the output match the rules your system was designed around? A voice or domain contract. Does the output match the rules your system was designed around? A verification contract. If the output makes claims, can those claims be traced to a source? A deduplication check. Is this genuinely new? Or is the system recycling itself? An audit trail. When something fails, can you reconstruct what happened without rewriting the entire pipeline? In software, we learned not to deploy only because code exists. In agent systems, we need to learn not to ship just because the artifacts look complete. The problem is not your agent will fail. Your agent will fail. The problem is when your system farmers that failure nicely and ships it downstream. Here's what I want you to take away. Map your handoffs from input to the final output. Every arrow between two steps can be a place where the output gets corrupted. You don't have to fix all of them. You just have to know where they are. Pick the most expensive handoff. Not the most complex, most expensive. The one where bad data can cost you the most. A wrong claim published publicly, a broken schema that cascades to three downstream skills, a duplicate that errors your audience's trust. That's where your first gate goes. Make it say no. A gate which logs only warnings is not a gate. It's a suggestion. The gate needs to block the artifact from moving forward. That's the difference between an impressive demo and an operable system. Before you add another agent, add one boundary. Thank you for watching.

📌 文中提及的人物和组织

关键字: agentic-workflow ci-cd contract-testing quality-gate