AI落地:从狂热到务实——组织如何真正赢得AI竞赛 The Pragmatic Engineer 2026-02-24

AI前沿:好奇、炒作与现实的交织

我们今天将进行一次真正务实、脚踏实地的对话,探讨人工智能(AI)在组织内部的实际发展情况,我们能期待什么,以及代理(agents)如何正在改变游戏规则。为了进行这场对话,我想将大家的思绪带入太空。我看到了探索时代太空竞赛AI时代之间存在诸多相似之处。上周,我与一家小型初创公司的CTO兼联合创始人,以及一家大型、高度监管的银行的首席工程负责人进行了交流。我们花了整整15分钟,谈论我们正在构建的所有令人兴奋的项目,它们如何重新带来了编程的乐趣。我们涌现出无数想法,感觉根本无法足够快地构建出来,因为有太多东西需要学习和创造。这让我想起卡尔·萨根一句我非常喜爱的名言:“在某个地方,有些不可思议的事物正等待着被认知。”我认为这句话很好地捕捉了我们许多人对AI和正在进行的实验所感受到的那种可能性。

这种感觉与我们在太空探索时代,奔向月球、奔向火星时所感受到的如出一辙。然而,这并非没有质疑。为什么要在解决地球上的诸多问题时,花费巨资进行实验并奔向月球?我们充满了惊奇,但也伴随着巨大的怀疑,因为太空探索不仅仅关乎科学,它还涉及全球、经济和政治层面。太空并非解决我们人类所有问题的万能药。但我们也不能否认,当人类登上月球时,那对全人类而言是一个极其关键、具有里程碑意义的时刻,充满了奇迹,令世界为之惊叹。那是在重新定义“可能性”。

同样地,我们对AI也充满了惊奇、乐观和希望。我们可以谈论生产力的巨大提升,以及围绕着“100%的生产力提升”和“所有代码都由AI编写”的种种炒作。我们看到了“单人独资的十亿美元初创公司”的潜力,即一个人拥有一个想法,并配备一支由代理组成的军队。外界充满了乐观情绪。但同样,也存在大量的怀疑。在企业界,人们对AI的实际经济影响持怀疑态度,考虑到其高昂的成本和环境影响。许多研究也对AI的实际生产力影响表示怀疑。在某些情况下,AI能极大地加速进程;而在其他情况下,它反而会拖慢我们的脚步,阻碍我们前进。我们很难分辨什么是真实的。

但随着技术的发展,在探索宇宙的同时,认识到我们仍有地球上的问题需要解决,这种好奇心和惊奇感是好的,也是自然的。我们必须学会平衡这种惊奇感和好奇心,同时也要承认我们生活在现实中,需要让双脚牢牢地站在这片土地上。我们需要理解这些实验将如何应用于日常的公司运营,我们究竟将如何改善我们周围的世界。我们需要保持那份惊奇感,同时也要用务实精神来平衡它,并通过数据分析来超越那些浮夸的炒作。

Original English Today I wanted to have a really pragmatic and downto-earth conversation about AI, what is actually happening in our organizations, what you can expect to happen um and how agents are changing the game. And I thought in order to have this really pragmatic down-to-earth conversation, I wanted to take us to space.

I do see a lot of parallels between the age of exploration and the space race and the age of AI. So last week I was talking with a CTO co-founder of a small startup. I was also talking with a principal engineering lead at a very big bank, highly regulated and we sat for a solid 15 minutes talking about all the cool stuff that we were building, how it's brought back joy, the joy of coding, and we just had all of these ideas and it seems like we couldn't build fast enough. There is so much to learn and so much to build. And it reminds me of this really lovely quote from Carl Sean that I love that somewhere something incredible is waiting to be known. And I think that this quote really captures what a lot of us are feeling about AI and about the experimentation and just the possibility that is out there.

This is the same feeling that we had with the age of space exploration, going to the moon, going to Mars. But it didn't come without skepticism. Why spend all of this money experimenting and going to the moon when we had lots of problems to solve here on Earth? We had a lot of wonder, but we also had a lot of skepticism because space exploration wasn't just about science. It was also global. It was economic. It was political. Space wasn't a silver bullet to solve all of the problems that we had with humanity. But we also can't deny that when a man landed on the moon that it was a very pivotal defining moment for all of humankind and had a sense of wonder and had the world in awe. It was about redefining what was possible.

And similarly, we have a lot of wonder and a lot of optimism and a lot of promise about AI. We can talk about productivity boosts and all of the hype around, you know, 100% productivity boost and all of our code being written by AI. We have the promise of perhaps the first singleperson billion-dollar startup with a person with an idea and an army of agents. There's a lot of optimism out there. Similarly, there's a lot of skepticism. There's a lot of skepticism in the corporate world about the real economic impact of AI given how expensive it is, given the environmental impact. There's also a lot of skepticism in a lot of different studies about the real productivity impact. In certain circumstances, it can be really um it can really accelerate and in other circumstances it can actually slow us down and get in our way. It's hard to know what's real.

But as technology changes, it is really good and fine to have that sense of wonder of exploring the universe while also realizing that we have problems here on Earth to solve. We have to learn how to balance that sense of wonder and curiosity with the acknowledgment that we are living in reality and we need to keep our our feet firmly planted on this earth. We need to understand how these experiments are actually going to apply to everyday companies. How are we actually going to improve the world around us? We need to keep the sense of wonder while also balancing it with pragmatism and beating the hype by looking at data. And so that's what I want to do right now. I'm going to share some brand new AI industry benchmarks with you. This is new data that no one has ever seen ever before in the world. Okay, it's coming right now to you. Um, I just pulled these down. I guess the the static team has seen them because they they saw the preview of my slides, but aside from them, no one has ever seen it. Um, this is though not really surprising because a lot of these numbers have not changed very much from the last quarter.

衡量AI的实际影响:基准数据与应用洞察

现在,让我们深入了解一些最新的行业基准数据,这些数据是首次对外公布。我们分析了来自超过450家公司、共计121,000名开发者的样本数据,收集时间为2025年11月至2026年2月1日。数据显示,约92.6%的开发者至少每月使用一次AI编码助手来完成工作,而约75%的开发者每周至少使用一次。需要注意的是,开发者对“AI编码助手”的定义较为宽泛,通常包括CodexCopilotClaude等工具,但不一定特指ChatGPT。

在衡量生产力影响的关键指标——时间节省方面,虽然它不是唯一的衡量标准,但它是一个重要的领先指标。数据显示,平均每位开发者每周因使用AI工具而节省约4.08小时。这个数字与2025年第二季度和第四季度的结果(约3.6-3.7小时)相比,变化不大,大致稳定在每周4小时左右。这大致相当于10%的生产力提升,与一些研究(如Google去年的文章)引用的数据相当。在过去几个季度里,这个数字并未发生戏剧性变化。

然而,正在快速变化且显著增长的是AI编写并成功合并的代码比例。在同一时间段内(2025年11月至2026年2月),对约42,600名开发者的数据分析显示,行业平均有26.9%的代码被AI编写并合并到生产环境中,且无需进行大量人工干预。这一比例较上一季度的22%有了显著提升,表明AI在代码生成和集成方面的能力正在快速增强。目前,每日使用AI的开发者比例已超过30%,这意味着近三分之一的代码是由AI生成并通过了代码审查,最终进入了面向客户的环境。

AI在开发者入职(onboarding)方面展现了巨大的潜力。我的初步判断是,AI将成为一个出色的入职工具,能帮助新员工更快地获取信息。数据显示,从2024年第一季度到2025年第四季度,开发者完成第10个Pull Request(PR)的平均时间缩短了一半。第10个PR通常被视为一个重要的入职里程碑。当我们将AI使用率的提升与入职时间的缩短相关联时,会看到一条非常漂亮的增长曲线。AI在加速新员工融入代码库方面表现出色。这不仅限于新招聘的员工,对于调动项目或非工程师转岗入职也同样有效。MicrosoftBrian Hulcat(《开发者生产力太空框架》的合著者)的研究表明,这种通过AI加速的“第10个PR”表现,能够持续影响工程师长达两年。因此,更快的入职速度带来的生产力提升并非昙花一现,而是能长期保持。

尽管我们看到了这些行业基准和平均数据,但必须强调,平均值不代表典型值。随着数据分布的离散化,平均值可能无法反映大多数人的实际体验。AI的使用体验在每个公司都截然不同,因为每个公司都有其独特的问题和文化。这种不均衡的影响,就像宇宙大爆炸一样,将组织推向了不同的方向。AI是加速器和乘数,它会根据组织原有的基础和方向,将其推向不同的极端。例如,在质量方面,一些组织正面临客户面向的事件数量增加一倍的情况,而另一些组织则经历了50%的事件数量减少。前者可能因为AI放大了原有问题,后者则通过AI优化了系统,实现了更快、更高质量的交付。

Original English Okay, it's coming right now to you. Um, I just pulled these down. I guess the the static team has seen them because they they saw the preview of my slides, but aside from them, no one has ever seen it. Um, this is though not really surprising because a lot of these numbers have not changed very much from the last quarter. So, what we're looking at here is a sample of 121,000 developers at over 450 companies. This data was pulled from November through February 1st, 2026. I I really just did this. Um, we're sitting around 92.6 of developers are using an AI coding assistant at least once a month to get their work done. And about 75% of developers are using an AI coding assistant at least once a week. When I say a AI coding assistant, most developers define that as cursor, Codex, Copi, you know, Claude, not ChatgPT necessarily. Um, but it is a bit open-ended, so keep that in mind.

When it comes to time savings, time savings is not the only measure of productivity impact, but it is an important signal. It's a good leading indicator. We're sitting around 4.08 uh self-reported hours saved due to AI tool usage per week per developer. This is not all too different from the number that came in Q2 of 2025. And then the number for Q4 of 2025 was about 3 uh six or seven. So this is kind of hovering around the 4 hour mark. And there's been a few articles, for example, from Google in the last year citing about a 10% productivity increase. And if we look at it in terms of time savings, we're kind of hovering around that 10% mark. It hasn't changed dramatically um over the last few quarters.

What is changing and what is moving up very quickly is the amount of code getting merged upstream or in a customer-f acing environment that was written by AI that was merged without significant human intervention. We call that AI authored code. And in a sample of around 42,600 developers from that same time frame, uh, November 1st to February 1st, 2026, we're at about 26.9% industrywide for all of these developers. That's how much code is hitting production that was AI authored. This is moving up from 22% in the last quarter, which is actually a pretty significant change quarter over quarter. And we can see that daily users of AI have crested over that 30% mark. So almost a third of their code is being written by AI that is actually being merged, passing through code review and getting into a customer-f acing environment.

One of my favorite use cases for applying AI is to onboarding. And I had a bit of a hunch that AI was going to be a great tool for onboarding, helping connect people with information earlier and sooner. And I have all of this data and I thought, let me look at this quarter over quarter. And in fact, if we look at Q1 of 2024 all the way over here on the left side and fast forward to Q4 of 2025, we have about a half uh we have a half reduction in onboarding time. This is looking at the time to 10th PR. So by the time a developer hits their 10th PR, that's a pretty important onboarding milestone that the industry has mostly aligned on in terms of onboarding and that has been cut in half now. And when we correlate that with the uptick of AI usage, it makes a really pretty graph. Uh AI is fantastic for onboarding. And this is not just brand new hires to your company. We've also seen plenty of evidence that this is for engineers who are moving projects or even non-engineers coming onboarding into projects. What's really important about this number is that there was a separate study done um by Brian Hulcat at Microsoft. He's the co-author of the space framework of developer productivity and they found in Microsoft's context that the time to 10th PR actually that performance sticks with an engineer for their first two years of tenure. So if you onboard faster that productivity gain isn't just onboarding it actually sticks with them for at least two years after they have started at the company. So this is a very important and significant uh trend that we're seeing here with using AI to connect developers, reduce cognitive load, and get them onboarded more quickly into their code bases.

One thing that's really important for me to call out, although I have just shared with you industry benchmarks, averages are just math. And as the polls move further away from each other, the average stays the same. Average does not mean typical. It does not mean what is going to happen to you. And it doesn't mean what a common experience is. One thing that is absolutely true, one thing that is common is that there is no typical experience with AI. There is no typical experience with AI. It is it is extremely different in every single company because every company has their own problems and their own culture.

This uneven impact can take us back to space for just a minute. So we can go back to the origins of the universe. We had the big bang and there was this massive release of energy and as this energy released the time the the space and time in between objects grows bigger right things are moving apart and for a lot of us the emergence of AI and AI coming into our organizations and in the industry has felt a lot like this big bang we've had this explosive release of energy in the center of our world and things keep moving apart organizational performance is multi-dimensional and these organizations are just going off into different extremes teams based on what they were doing before. AI is an accelerator. It's a multiplier and it is moving organizations off in different directions. The best example I can share with you of this is quality. Okay, so in this case, this is not every organization, but some organizations are facing twice as many customer-f acing incidents and this is from a sample of over 67,000 developers from Q1. So that same time frame of uh November to February. So just looking in that time frame organizations are experiencing twice as many customerf acing incidents at the same time at the same time companies are also experiencing 50% fewer incidents. So some companies have used AI they have a really healthy system it has amplified that system they are seeing fewer incidents they're moving faster they are accelerating with higher quality higher code maintainability higher change confidence. On the other side though, blasting off into the other part of the universe, we have organizations who were dysfunctional already. No, they're more dysfunctional. They're dysfunctional and dysfunctional faster. Okay.

代理(Agents)与组织系统:迈向真正影响力的路径

我们正处在一个宇宙不断扩张的时代,这得益于代理(agents)及其驱动的自主工作流(agentic workflows)的应用。我们的宇宙正在变得更加广阔,随之而来的是各种承诺和炒作,但同样增长的还有其潜力。让我们回到登月。登月所代表的终极炒作是,我们都将在此时此刻生活在月球上,乘坐像《杰森一家》那样的飞行汽车。同样,在这里,我们也看到了一些“疯狂”的想法。例如,如果你用过Gas Town,你会发现它提供了极其丰富且充满创意的可能性。Gas Town对我来说无限有趣,有太多值得探索的方面。但请注意,Gas Town目前可能有些“不受约束”(unhinged),不建议在企业环境中直接使用。我们还有OpenClawMaltbotClawbot,以及Ralph loops等各种工具。这些实验带来了巨大的乐趣和创造力。然而,我个人在家制作一个匹配指甲油颜色方案的应用,与一家跨国银行能够通过AI改变其收入,这是截然不同的。

在一次与Martin FowlerKent Beck等行业先驱的研讨会上,我们花了大量时间试图将AI的应用与实际的盈利能力(P&L)联系起来。最终,我们聚焦于一个核心问题:创新的价值是什么?即使我并不居住在月球上,去月球的探索是否仍然有价值?我的观点是,创新的确是有价值的。但这可能进入一个模糊的区域,因为我们身处商业世界,而非仅仅是为人类福祉而行动的社会。我们必须在经济背景下进行考量,这会变得有些棘手。

当我们回顾那句“在某个地方,有些不可思议的事物正等待着被认知”时,AI和太空探索都蕴含着一种探索的惊奇感,它们都令人兴奋。但登月并非意味着我们都需要居住在月球上。事实上,登月以及所有这些“疯狂”的探索,其根本目的是改善地球上的生活。我们利用太空探索的奇迹,将其应用于我们当时在地球上遇到的系统级问题。并非每个人都想住在月球上,但我们因此获得了太阳镜、太空毯、条形码、石英手表等众多技术和改进,这些都源于那场伟大的太空探索时代。尽管我们没有居住在月球,但我们依然吸收了其中的经验教训,并将其应用到地球上的系统建设中。

因此,当我们思考**自主工作流(agentic workflows)**时,代理极大地扩展了我们能够构建什么、如何构建以及为谁构建的可能性。并非每个人都要去月球,而且不去月球也完全没问题。并非每个人都会在企业环境中每天使用Gas Town构建“疯狂”的东西,这同样是可以接受的。因为这些实验有助于拓展可能性的边界,并帮助我们以新的方式思考问题。

目前,代理在行业中的应用正在兴起。尽管如此,真正领先并为代理用例配备良好遥测数据的公司并不多。一个较小的样本显示,在6家提前部署了代理工作流遥测数据的公司中,约有80%的开发者每周至少使用一次代理工作流,超过50%的开发者每天都在使用。以Codex为例,其桌面应用发布后,下载量已超过百万,并且用户增长迅速。OpenAI内部95%的开发者使用Codex进行开发,且使用Codex的开发者比使用其他AI工具的开发者每周多交付约60%的PR。这有力地证明了像代理工作流这样的新工具所带来的高潜力。

让我们看一个非AI初创公司的例子:Haven Headache and Migraine Center。这家位于旧金山的公司,旨在解决“能否通过Zoom治疗头痛”的问题。在医疗保健领域,区分代理用于“持久代码”还是“一次性代码”至关重要。Haven利用代理快速原型化新的患者工作流,例如通过Ralph loops构建患者门户。他们获得的是高质量的原型,而非“一次性AI垃圾”。此外,他们还通过训练一个符合HIPAA标准的模型,处理数十万份症状日志,以改进患者护理标准。用户可以通过短信记录症状,系统会据此安排药物续订或随访。其结果是,客户满意度是行业平均水平的三倍,并且患者的头痛天数和严重程度均显著降低。

在企业领域,大型企业也在积极试验代理工作流。一家制造企业使用C-pilotClaude构建内部开发者门户,以加速开发者入职。Cisco有18,000名工程师日常使用Codex,用于复杂的迁移和代码审查,将代码审查时间减少了50%。JP Morgan ChaseMAFA(多代理框架用于注释)论文也展示了其构建一个类似Gas Town的代理业务,每个代理负责特定任务,并引入了代理间的共识机制来验证和校准输出。我们相信,代理间的共识将在2026年成为一个重要的待解决问题。

在一次关于软件开发未来的研讨会上,我们得出的结论是:AI本身并不能解决组织系统性问题。只有当我们将其应用于系统性问题时,它才能发挥作用,这意味着首先要承认系统性问题的存在。AI并非神奇的银弹。尽管存在Gas Town这样的工具,尽管宇宙充满好奇和惊奇,但我们不能忽视一个关键点:组织受限于人类和系统层面的问题。我们对任何声称能在不解决这些根本性约束的情况下提升组织绩效的技术都持怀疑态度。如果我们不解决系统性问题,我们将只是把这些问题带到太空。我们无法仅仅去月球,就期望污染、垃圾和交通拥堵问题会消失。

因此,关键问题不是如何殖民火星,而是如何通过代理和AI获得真正的组织影响力。在研讨会上,我们还讨论了组织成功的共同因素:目标与衡量。盲目地向所有开发者发放许可证并期望奇迹发生是无效的。AI创新必须指向具体问题,设定明确目标,并衡量进展。正如斯波克所说:“事实不足总是招致危险。”我们需要数据。

为了解决开发者生产力工程卓越性的难题,我们提出了AI测量框架。该框架旨在追踪AI的使用、采用和利用情况,并将其转化为实际的组织影响,如速度、开发者体验、质量和创新率。同时,我们必须关注成本,评估投资回报。

其次,**开发者体验(Developer Experience, Devex)**比以往任何时候都重要。一个非常规的建议是:将任何关于开发者体验的讨论,都包装成“代理体验”(Agent Experience),这样更容易获得资源。这并非玩笑,而是因为开发者体验、反馈循环、清晰的服务定义、良好的文档和快速的CI/CD,这些我们呼吁了几十年的事项,恰恰是AI取得成功的关键。我们需要扎实的测试和质量实践,以及出色的文档,这些对代理工作流至关重要。令人遗憾的是,我们曾不愿为人类工程师投入,却愿意为“机器人工程师”投入。但这就是我们所处的现实,我们应抓住机遇。

数据显示,虽然AI能节省约4小时的编码时间,但这不足以弥补糟糕的会议文化、频繁的打断、非计划性工作或系统故障。AI可以帮助解决这些问题,但它本身无法完全取代良好的组织管理。真正能带来长远效益的是,将AI应用于解决这些系统性问题:能否用AI减少会议频率?能否用AI改进CI等待时间?能否用AI减少开发环境的繁琐操作?那些成功的组织正将Devex置于中心,并将AI视为解决系统级问题的工具。

最后,要实现组织级成果(如收入、利润、上市时间),必须将AI视为一个组织性问题,而非仅仅是开发者个人在桌前解决的问题。它必须应用于跨越整个价值流的工作流程。MIT的一项研究表明,组织在AI采纳上面临的障碍并非技术性问题,而是变革管理缺乏高层支持(领导者自己不使用AI工具)、糟糕的用户体验以及不明确的期望

如果这些听起来很熟悉,并且您的组织可能需要改进,可以参考DORA AI能力模型Thoughtworks Thor框架。这些都是经过充分研究、获得行业支持的AI就绪度模型,可以帮助您与领导层沟通,或进行内部审计,确保组织已做好准备,以充分利用AI带来的实验成果。

最重要的一点是,那些在AI方面表现出色的组织,正在通过解决真实的客户问题来进行实验。太空探索固然伟大,但让整个组织都去“探索火星”是不可持续的,成本过高,且会分散核心业务的注意力。因此,请保持实验精神,但要将其高度聚焦于真实的客户问题,这才是看到组织成果的关键。

“在某个地方,有些不可思议的事物正等待着被认知。”AI和代理正在加速这一进程,扩展我们的宇宙。我们正处在一个探索时代。我希望大家在接下来的会议中,能找到平衡点:既要保持对未知的好奇与敬畏,追求宏大的目标(如登月),也要理解我们必须解决地球上的现实问题。请保持脚踏实地,保持审慎,保持人性,最重要的是,保持务实。谢谢大家。

Original English And so that's what I want to do right now. I'm going to share some brand new AI industry benchmarks with you. This is new data that no one has ever seen ever before in the world. Okay, it's coming right now to you. Um, I just pulled these down. I guess the the static team has seen them because they they saw the preview of my slides, but aside from them, no one has ever seen it. Um, this is though not really surprising because a lot of these numbers have not changed very much from the last quarter. So, what we're looking at here is a sample of 121,000 developers at over 450 companies. This data was pulled from November through February 1st, 2026. I I really just did this. Um, we're sitting around 92.6 of developers are using an AI coding assistant at least once a month to get their work done. And about 75% of developers are using an AI coding assistant at least once a week. When I say a AI coding assistant, most developers define that as cursor, Codex, Copi, you know, Claude, not ChatgPT necessarily. Um, but it is a bit open-ended, so keep that in mind.

When it comes to time savings, time savings is not the only measure of productivity impact, but it is an important signal. It's a good leading indicator. We're sitting around 4.08 uh self-reported hours saved due to AI tool usage per week per developer. This is not all too different from the number that came in Q2 of 2025. And then the number for Q4 of 2025 was about 3 uh six or seven. So this is kind of hovering around the 4 hour mark. And there's been a few articles, for example, from Google in the last year citing about a 10% productivity increase. And if we look at it in terms of time savings, we're kind of hovering around that 10% mark. It hasn't changed dramatically um over the last few quarters.

What is changing and what is moving up very quickly is the amount of code getting merged upstream or in a customer-f acing environment that was written by AI that was merged without significant human intervention. We call that AI authored code. And in a sample of around 42,600 developers from that same time frame, uh, November 1st to February 1st, 2026, we're at about 26.9% industrywide for all of these developers. That's how much code is hitting production that was AI authored. This is moving up from 22% in the last quarter, which is actually a pretty significant change quarter over quarter. And we can see that daily users of AI have crested over that 30% mark. So almost a third of their code is being written by AI that is actually being merged, passing through code review and getting into a customer-f acing environment.

One of my favorite use cases for applying AI is to onboarding. And I had a bit of a hunch that AI was going to be a great tool for onboarding, helping connect people with information earlier and sooner. And I have all of this data and I thought, let me look at this quarter over quarter. And in fact, if we look at Q1 of 2024 all the way over here on the left side and fast forward to Q4 of 2025, we have about a half uh we have a half reduction in onboarding time. This is looking at the time to 10th PR. So by the time a developer hits their 10th PR, that's a pretty important onboarding milestone that the industry has mostly aligned on in terms of onboarding and that has been cut in half now. And when we correlate that with the uptick of AI usage, it makes a really pretty graph. Uh AI is fantastic for onboarding. And this is not just brand new hires to your company. We've also seen plenty of evidence that this is for engineers who are moving projects or even non-engineers coming onboarding into projects. What's really important about this number is that there was a separate study done um by Brian Hulcat at Microsoft. He's the co-author of the space framework of developer productivity and they found in Microsoft's context that the time to 10th PR actually that performance sticks with an engineer for their first two years of tenure. So if you onboard faster that productivity gain isn't just onboarding it actually sticks with them for at least two years after they have started at the company. So this is a very important and significant uh trend that we're seeing here with using AI to connect developers, reduce cognitive load, and get them onboarded more quickly into their code bases.

One thing that's really important for me to call out, although I have just shared with you industry benchmarks, averages are just math. And as the polls move further away from each other, the average stays the same. Average does not mean typical. It does not mean what is going to happen to you. And it doesn't mean what a common experience is. One thing that is absolutely true, one thing that is common is that there is no typical experience with AI. There is no typical experience with AI. It is it is extremely different in every single company because every company has their own problems and their own culture.

This uneven impact can take us back to space for just a minute. So we can go back to the origins of the universe. We had the big bang and there was this massive release of energy and as this energy released the time the the space and time in between objects grows bigger right things are moving apart and for a lot of us the emergence of AI and AI coming into our organizations and in the industry has felt a lot like this big bang we've had this explosive release of energy in the center of our world and things keep moving apart organizational performance is multi-dimensional and these organizations are just going off into different extremes teams based on what they were doing before. AI is an accelerator. It's a multiplier and it is moving organizations off in different directions. The best example I can share with you of this is quality. Okay, so in this case, this is not every organization, but some organizations are facing twice as many customer-f acing incidents and this is from a sample of over 67,000 developers from Q1. So that same time frame of uh November to February. So just looking in that time frame organizations are experiencing twice as many customerf acing incidents at the same time at the same time companies are also experiencing 50% fewer incidents. So some companies have used AI they have a really healthy system it has amplified that system they are seeing fewer incidents they're moving faster they are accelerating with higher quality higher code maintainability higher change confidence. On the other side though, blasting off into the other part of the universe, we have organizations who were dysfunctional already. No, they're more dysfunctional. They're dysfunctional and dysfunctional faster. Okay.

Similarly to this uneven impact, organizations are seeing really uneven results like economically from using AI. there are a lot of steep drop off drop offs when it comes to using AI in a pilot context to production and then actually trying to tie it to profit. This is from uh an MIT study that was published in July of 2025 called the Gen AI divide. And what the study concluded they did a survey of 152 organizations was that right now where we are in the industry is that we have really high adoption, right? That 92.6 number. Um DORA also does its own research. We're hovering around that 90% adoption number. high adoption but actually low transformation because as it turns out transformation is really uncomfortable and organizations that were ready to give up on the cloud transformation on the agile transformation are also giving up on their AI transformations. It is really really difficult to look at your whole organization and look at the problems and think we got to change something about this and that is what organizations need to do in order to actually see change to their bottom line.

All of this to say back to my previous point we have 92.6 adop uh percent adoption among developers in our industry but adoption doesn't mean impact. Using the tool doesn't mean that it's going to actually advance your organization or do anything. It is an organizational problem that needs organizational change management. But that's not really what we were promised with all of the hype was like, hey, experiment with AI and then something happens and then we profit. What happens though is that these tools were primarily deployed into individual coding tasks. And what this MIT study found in this high adoption low transformation is that when we apply it only to the surface area of a developer sitting at their desk there is a very very low ceiling of productivity gain. This is an organizational problem. If we want organizational results we have to think about it on an organizational level not on a coding task level.

Fortunately, our universe is expanding right now and that expanding is coming through the use of agents in agentic workflows. Our universe is getting bigger and so are all of the promises and all of the hype, but so is the possibility. So, let's go back to the moon landing, right? Like the ultimate hype was that we're all going to be living on the moon by now in flying cars like jetson style. Um, similarly here we have a little bit of like crazy ideas. Um, Gas Town, if any of you have used it, um, there there's just like there's so much crazy stuff to do right now. Um, Gas Town is infinitely interesting to me. There are so many interesting things. Um, disclaimer, don't use Gas Town. It is unhinged. Um, we've got OpenClaw, Maltbot, Clawbot, whatever it's called. We've got Ralph loops. We've got all the stuff, right? There is so much experimentation and so much fun. It's just really fun to build. Um, but me building my nail polish matching like color scheme app while I'm sitting at the nail salon is not the same as a multinational bank being able to change their revenue because of AI. Those are really different things. Um, and I was at this retreat with Martin Fowler and Kent who are I think back there. Hello. We'll talk about that a bit more later. We spent a lot of time trying to connect AI and the use of AI to bottom line, to profit, to P&L. And interestingly, kind of where we landed at the end was this question of like what is the value of innovation? Was it still valuable to go to the moon even though I'm not really located on the moon right now? And I would argue that yes, it is valuable to innovate. And that can get into some murky area because this is a business, right? This isn't just society and and doing things for the good of humankind. we have to do them in an economic context and that can get a little bit tricky.

So when we think about this quote something uh somewhere something incredible is waiting to be known. There is a sense of wonder and AI and space are both the age of exploration and it is so exciting. But the point of going to the moon wasn't that we all need to live on the moon. In fact, the point of going to the moon and the point of exploring and doing all this crazy stuff was to improve life on Earth. It was to use the space exploration and all of this wonder to apply it to the systems level problems that we had back on Earth. Not everyone wants to live on the moon. Um, but we have sunglasses, we have space blankets, we have barcodes, we have quartz watches. We have so much technology and so many improvements back on Earth because of this crazy age of exploration where we all went to space. Even though we're not living on the moon, we've still used the lessons and applied it to our systems back here on Earth.

And so thinking about agentic workflows, agents expand the possibilities of what we can build, how we can build it, and who we can build it for. Not everyone goes to the moon, and it's okay not to go to the moon. Not everyone is going to be building crazy stuff with Gas Town every day in in your enterprise context. And that's also okay because the experimentation helps push the boundary of what's possible and helps us think about solving problems in new ways.

So, let's talk a little bit about how agents are being used in the industry right now. Again, this is new data that I'm sharing for the first time here. Um, agentic use is on the rise. There's not a lot of companies, honestly, that are so far ahead of the curve that they're already uh instrumenting their agentic use cases with really good telemetry. This sample is a little bit smaller. It's around 3,000 developers at six companies. Keep in mind, these companies are ahead of the curve. They're already instrumenting their agentic workflows with telemetry. Um, we have about 80% of developers using these agentic workflows at least once a week with over 50% using agentic workflows every single day to get their work done. We talked about codecs I think in the previous panel. So on February 2nd, the Codeex desktop app was released and since then there's been a million over a million downloads by now. I got this data yesterday. I'm sure it's quite different by now. There's been a 60% growth in users just in the last week. Um they also launched uh GPT 5.3 codecs uh last Thursday. They're processing trillions of tokens per week. Internally at OpenAI, 95% of developers are using codecs to ship stuff. And of the developers who are using codecs versus other AI tools, the developers who use codecs are shipping about 60% more PRs per week, which is very interesting. a data point, not the only data point, but it just speaks to the very high ceiling, the high possibility, the sense of wonder that we have with building all of the stuff with cool new tools like Agentic Workflows.

I want to bring it back to a nonAI startup though. Um, I want to highlight Haven Headache and Migraine Center. So, this is a company that's based here in San Francisco, actually just a few blocks away. Haven set out to answer the question, can we solve headaches with Zoom? And it turns out you can. Um, so if you're a headache sufferer, this might be useful for you to to learn about. In healthcare, it's really really uh crucial for Haven and their development team to distinguish between using agents for durable code or disposable code. One of the things that they're doing that's very cool since they are a disruptor, they are a small startup is using um using agentic workflows to rapidly prototype new custom uh like new patient workflows. So they're working on a patient portal building with Ralph loops taking uh linear and Figma artifacts changing it into a PRD uh you know spitting that out in JSON and then just having Ralph loops run. What they're getting though isn't garbage disposable AI slop. What they're getting is really high quality prototypes with really excellent documentation, excellent tests, much higher quality at a way faster rate than they would have um if they would have built it by hand the oldfashioned way. The other thing that they're doing that I really admire is improving the standard of care for their patients by training a HIPACO compliant model on hundreds of thousands of symptom logs. So Haven meets you where you're at. You get a text message, you can log your symptoms and then they can um instrument your care, figure out what needs to happen from there. So they're training a hypocmplant model on hundreds of thousands of these messages so that those messages can be routed to, you know, medication refill or schedule follow-up appointment just meets you where you are. And the result of this is that they have 3x the industry average in customer satisfaction for a healthcare tool like this, but also real real meaningful clinical outcomes. So their patients have fewer headache days per month and also the severity of their headaches is much uh much less severe. So good job Haven. In the enterprise, there are lots of examples of big enterprise companies experimenting with agent workflows. So there's an enterprise manufacturing company that's using it for solely internal developer purposes. They used C-pilot and Claude to build out a dev portal to accelerate uh developer onboarding. At Cisco, there's 18,000 engineers using codecs daily. They're using the uh codecs for complex migrations and also code review leading to a 50% reduction in the amount of time it takes to do code review. There's a really cool paper as well by JP Morgan Chase's multi-agent framework for annotation, MAFA. If you Google that, you can find the source paper. It's really fascinating. Um, what they're doing is building out like a whole business of agents. So like a true multi- aent workflow similar to Gastown where each agent has a special job to do. What they're also doing in this model is introducing consensus among the agents. So they're taking all of these interactions and then they're annotating them. this was you know the intent what was it an FAQ what what were all these interactions the agents are annotating them and then there's another set of agents who are responsible for reranking and calibrating and validating the output and then of course we have to introduce consensus algorithms to the party because now we have multiple agents with maybe multiple different opinions about things um this is really fascinating and I believe consensus among agents is going to be a huge problem to solve in 2026

I spoke about this retreat. I was lucky enough to be invited by Martin Fowler and thoughtworks to the future of software development retreat celebrating the 25th anniversary of the agile manifesto of Gerge joined me. A few other folks who are here also joined me. We spent a day and a half up in the mountains talking about agents. That's really all we talked about about using agents responsibly, ethically, sustainably, how we can use them for organizations. And our conclusion even though there was so much interesting stuff, Steve Yaggi was there, we were working on Gas Town things, like there was a lot of experimentation happening, but the conclusion that we came to was that AI does not solve organizational systems problems. It only can do that when you apply AI to the system problem, which means you need to acknowledge that the system problem exists in the first place. AI is not a magic silver bullet. Even though things like Gastown exist, even though there is so much sense of curiosity and wonder in the universe, um we kind of had a sort of off-the cuff conversation. Uh Kent Beck, Steve and I were just catching up um outside uh of one of the sessions in between conversations. And here's sort of where we summarized our thoughts. Organizations are constrained by human and systems level problems. We remain skeptical of the promise of any technology to improve organizational performance without first addressing those human and systems level constraints. We remain skeptical and we also remain human because the risk is if we don't address the systems level problems, we will just take them to space with us. We will just take them to space with us. We're not actually going to solve the human factors that are the driving force behind all of the constraints that organizations have right now. We can apply AI to those problems, but we still need to solve them. We can't just go to the moon and expect that pollution and garbage and traffic aren't going to be a problem anymore.

And so the question is not how to colonize Mars, but the question is how to get real organizational impact with agents and AI. At this retreat, we also talked a lot about common factors that we see. What do we see organizations doing? What are the common patterns that is kind of like the secret to to winning? What do they have in common? The first one is that organizations who win with AI and are winning with AI have goals and they measure their progress against those goals. Spray and prey does not work. Spray and prey, what I mean by that is just giving all of your developers licenses and hoping for the best. It does not work. I can say that very very clearly. I have a lot of evidence that does not work. If you can point AI innovation and that experimentation to a problem, have a concrete goal and then measure if you're reaching that goal, that is what winning organizations are doing right now. Because as Spock has told us, insufficient facts always invite danger. We need to measure things. We need to have data. And I know this is something that's really difficult for a lot of organizations right now because developer productivity and engineering excellence are also really hard problems. And this is happening all at the intersection. So I have something that can help you if that is in uh a problem that you're facing in your organization. This is the AI measurement framework. This is a framework that I co-authored with Abby notto who's the CEO of DX. This complements our core four framework which some of you might have heard otherwise it's in the impact column here. What we're looking to do is track not just usage and adoption and utilization of AI but then also translate that into real organizational impact. Is this changing your speed, your developer experience, your quality, your innovation ratio? Those are really important questions to connect the adoption to impact. Finally, we have to look at the cost. Are we getting a good deal? Maybe some of us are for now. Um, and we need to understand as the cost of these tools keeps going up and up, is the investment the right one?

The second thing that is helping organizations win is that developer experience matters now more than ever. Here is a piece of very unconventional advice that I'll give you is just anything that you were going to talk about with your leadership team about developer experience. Just call it agent experience and you'll get money for it. It's funny but it works. Um, it works because developer experience, feedback loops, um, you know, clearly defined services, great documentation, fast CI, these are all things that we have been screaming about for decades, literally, and we've been begging for pennies from our organizations to please let us invest, please let us invest in developer experience. And we've been told no over and over again. Come to find out, in fact, these are the things that make AI really successful. We need to have really solid testing and quality practices. We need to have great documentation. These are critical for agentic workflows. It is disheartening that we didn't want to spend the money when it came to human engineers, but when it comes to robot engineers, we're okay with it. But that is the world that we live in and let's capitalize on our opportunity. So Devex matters more than ever. In fact, when we look at the data right now, remember we're hovering around that 4hour mark for time savings. when we look at all of the other factors of developer experience like AI time savings is not going to make up for bad meeting like bad meeting culture and lots of interruptions and um you know developers who are constantly being pulled out of their work unplanned work interruptions outages those kinds of things AI will not make up for that we can use AI to help solve that problem but AI in and of itself is not going to make up for it then when we look kind of in the bottom half build and test wait time toil and dev environment we put all that together We realize that just the time savings from coding task speed up isn't going to get us very far. But what will get us far is when we can take AI and point it at those problems. Can we use AI to help reduce meeting frequency? Can we use AI to improve CI weight time? Can we use AI to reduce dev environment toil? That is what winning organizations are doing right now. They are putting DevX at the center of their universe and seeing AI as a tool to fix systems level problems.

They're doing it also on an organizational level. If you want organizational outcomes like revenue, P&L, time to market, you have to think about AI on an as an organizational problem, not as an individual problem that your developer needs to solve at their desk. It has to apply to workflows that span entire value streams. back to that MIT study when we looked at the barriers to organizational adoption uh or the organizational barriers to AI adoption, they weren't technical. This wasn't about the models necessarily. It wasn't even about the tools that wrap the models. It was about things like change management or lack of executive sponsorship when you have an executive team saying go with AI, but they themselves have never cracked their laptop open and fired up windsurf or cloud code or codeex. um poor user experience, just very unclear expectations about AI. Those are the things that get in the way. If this sounds familiar to you and perhaps your organization could do a better job, there's two things that I want to point you to. Um the first one is the Dora AI capabilities model. These are models that kind of communicate and help you get ready for AI. So think about this as an AI readiness model or an AI capabilities model. This has in a crazy amount of data from organizations that Dora studies. They do a lot more than just the four key door metrics. Um, finding correlations between practices that organizations have and good outcomes with AI. So, if you use AI and have a good clear and communicated AI stance, you are going to do better organizationally than a company that does not have one. You can find this at dora.dev. It's the Dora AI capabilities model. There was just a new paper that came out last month. Last month, uh, Nathan is here who leads Dora over at Google Cloud. If you want to talk to him about this, he's probably the the guy. Um the other one is the thoughtworks forest framework. This is similar to the AI capabilities model kind of a different flavor on it. Um if you go to thoughtworks.com and look in their white papers you can read through this but these are both really solid wellressearched industrybacked AI readiness models to help convince your leadership team if you need that um or just help you do an internal audit of are we doing the right things to make ourselves ready to you know reap the benefits of all this experimentation.

The last thing is that organizations who are doing really well with AI right now are experimenting by solving real customer problems. Again, space exploration and going to Mars is great, but that is not sustainable for your whole entire organization to be experimenting with going to Mars. It just costs too much money. It distracts too much from the core business problem. It does not serve your customers. So, keep experimentation going. Other experimentation can be really laser focused on real customer problems that you have. And that is how you're going to see the organizational results.

Somewhere something incredible is waiting to be known. There is so much possibility of how we can build what we can build, who we can build it for right now with AI and agents are just accelerating this. They are expanding our universe. We are definitely in an age of exploration. The urge uh the thing I want to urge all of you to take with you into the rest of the sessions today is to find that balance between a sense of wonder and a sense of awe and aiming for Mars and aiming for your moon colony but also understanding that we need to solve the problems here on Earth and we have to live in this reality. So please stay grounded, stay skeptical, stay human, most of all stay pragmat stay pragmatic. Thank you all.

📌 文中提及的人物和组织

公司/组织: OpenAI, Microsoft, JP Morgan Chase, DORA

产品/模型: Codex