---
author: AI Engineer
date: '2026-09-30'
guest: ''
layout: post.njk
source: https://www.youtube.com/watch?v=Se8jHLliLXE
speaker: AI Engineer
tags:
- developer-productivity
- developer-experience
- software-metrics
- code-quality
- generative-ai
title: 400+ 企业数据复盘：AI 时代的工程生产力、代码质量与研发范式演进
summary: DX 副首席技术官 Justin Reok 基于 400 多家企业的海量实证数据，深度剖析了 AI 对研发效能的真实影响。数据显示，AI 虽驱动 PR 吞吐量与部署频率提升，却也引发代码变更失败率剧烈波动及代码量膨胀。破除‘初级开发者替代论’与孤立工具崇拜，企业亟需通过上下文工程、系统化指标监测与闭环工作流重构，将单点效率转化为端到端业务价值。
insight: ''
draft: true
series: ''
category: software-development
area: tech-engineering
project: []
people: []
companies_orgs:
- DX
products_models:
- GitHub Copilot
media_books: []
status: evergreen
---
### 破除速度迷思：度量体系与宏观采用趋势
在评估生成式 AI 对软件开发生命周期的实质影响时，我们必须首先建立严谨的度量底座。由 **DORA 指标**（DORA metrics: DevOps 研究与评估团队提出的交付效能四项核心指标）、**SPACE 框架**（SPACE framework: 涵盖满意度、性能、活动、沟通与效率的多维生产力评估框架）以及 **DevEx 框架**（DevEx framework: 聚焦心流状态、认知负荷与反馈循环的开发者体验度量体系）的原班学术专家构建的研究平台，为我们追踪 400 多家组织的底层工程流动提供了真实的数据样本。在度量语境下，诸如 **PR 吞吐量**（PR throughput: 单位时间内合并的拉取请求数量）和**部署频率**（Deployment frequency）等指标，本质上只是反映工作流运转速率的**代理指标**（Proxy metrics），并不能直接等同于最终的商业价值交付。
从宏观层面的技术采用率来看，企业引入生成式 AI 工具的比例整体呈上升态势，但呈现出显著的地域分化与行业异构性。北美地区对 AI 工具的采纳曲线平稳攀升，而欧洲市场在过去一个季度出现微幅回调，这与两地在监管合规约束、工程文化差异以及对 Token 算力成本的敏感度密切相关。更值得关注的是，工程团队内部存在高达 20% 到 25% 的**影子采用率**（Shadow adoption: 开发者绕过企业官方审批私自将个人 AI 账号接入工作流的现象）。在工程领导层对投资回报率仍持观望态度、等待第二季度更确凿报告的同时，一线开发者的实际渗透率早已远超管理层的统计账面，技术事实已经先于组织治理落地。
Original English Source
I'm Justin Reok. I am the Deputy CTO at DX. I'm involved in a lot of our research on developer experience and its relationship to productivity . And certainly over the past year, a lot of this research has focused on the impact of AI on the developer experience and basic aspects of productivity in an organization. We produce quarterly reports—our State of AI and Impact Reports— which analyze the vast amount of raw data we receive from our platform. Our research platform was created by the same experts who developed the DORA metrics, the SPACE framework, and the DevEx framework. Ultimately, it's a data collection platform that allows us to track a lot of different trends in the organization, including how we focus on the developer experience. Are we seeing results and improvements in the developer experience? And, of course, are we seeing results from how we use AI in our work processes? This session will be a little different from the one at 3:45 in the Executive Room—it focuses entirely on the raw data from our latest report. This is something like a preview. Here we look at more organizational trends, how our work processes are changing, based on the data we see. So, let's start with the main thing: let's see what we see in terms of the impact on development speed. If anyone is familiar with various speed metrics such as PR throughput and deployment rate, you hopefully understand the caveat that these metrics are not perfect. These are proxy metrics for understanding how work flows through the organization. They do not fully reflect value creation. We will also look at some metrics that are closer to this. But let's try to understand what this technology does in terms of pure speed in an organization. Here is our DORA metric for deployment frequency. Who is familiar with DORA metrics? Great, so you know this well: DORA deployment frequency measures how often an organization delivers changes to production. Our data covers about 400 organizations, and over the past year we have seen steady growth in deployment frequency. This growth is real, but it varies by organization. But in general, organizations that use AI tools are, on average, experiencing steady growth in deployment frequency. You can see it on this chart—we have a slight dip here, but overall there is a steady upward trend. Another indicator is the adoption rate of AI tools in organizations. We see that adoption is growing, but there are certain patterns. There are always people who doubt that AI will become an integral part of work, but, you know, the data shows that this number is steadily increasing, okay? Um, the trends vary a bit depending on the region. Um, we see that there is an upward trend in North America . In Europe, we see a slight decline over the last quarter. Um, I think, you know, there are a few reasons for that. I mean, obviously the way work is done in these regions is different. The way we spend money on these technologies and evaluate token costs varies. The regulation is also different. But if you look at North America, adoption is growing, and we know from other studies that shadow adoption is likely in the 20 to 25% range. So even where it looks like things have stabilized, adoption is likely still growing. A lot of organizations are still waiting to see the results. When I talk to our customers, they are all waiting for the results of the second quarter to understand what AI will bring them. But developers, of course, are actively using these tools.
---
### 认知与现实断层：交付速率与生产力假象
在研发效能的感知与客观数据之间，存在着一条耐人寻味的认知断层。调研显示，一线工程师普遍主观感觉交付速度获得了极大提升，自评在 AI 辅助下的编码产出突飞猛进。然而，将这种**主观速度感知**与实际代码库的流转速率相对比时，客观指标的涨幅却远不如感知那般激进。行业早期流传的“Copilot 能提升 55% 编码效率”的宣传结论，在严肃的企业级真实场景中遭遇了现实重击；在针对 16 名资深工程师开展的微观受控实验中，开发者在复杂业务系统的维护任务中，其综合生产力甚至在统计意义上下降了 18%。这一反差揭示出：单点代码生成的敏捷感，并不等同于全链路交付周期的缩短。
深入分析这一背离现象的根源，在于传统流水线中存在的结构性瓶颈并未因前台代码生成加速而消解。**前置时间**（Lead time: 任务从提出、设计、开发、评审到上线发布的全程耗时）中真正的耗时大头，往往沉淀在跨团队协作、架构评审、端到端集成测试以及发布阻滞上。当 AI 让开发阶段的产出瞬间翻倍，下游的**代码审查**（Code review）与验证管道却迅速过载。这表明孤立地在编码阶段注入生成式能力，若不配套优化组织级的流转机制，不仅无法达成端到端的效能跃升，反而会制造更严重的在制品堆积与系统拥堵。
Original English Source
Um, the perception of delivery speed. Developers report an improvement in their speed, but if we compare this data with real speed metrics, the improvement is not as large as perceived. This is quite interesting. Many of you have probably heard of a 55% increase in developer speed, right? It's a very famous number from early Copilot studies that has been widely discussed. But given that it's been a full year, and all the investment and spending on AI, it's interesting that we're not seeing that 55% increase. It's actually a bit worse. There's a new study that just came out, and it's not a very good study, it was done on a small sample of 16 people. But what's interesting is that among the 16 engineers in this small, flawed study, productivity dropped by 18%. This shows that measuring productivity in software development is very difficult, and this is well-known. If you ask a developer, "Are you moving faster?" they will answer, "Yes, I am moving much faster." But when we measure it, the real speed increase is not as significant as the perceived one. Okay, let's look at quality metrics. Let's see what happens to the quality of the software we write. The first metric we will look at is the DORA indicator for the rate of failed changes.
---
### 质量波动性放大：变更失败与代码膨胀风险
当我们将观察视角转向系统质量维度时，数据展现出前所未有的剧烈震荡。作为核心质量基准的 DORA 指标——**变更失败率**（Change failure rate: 部署到生产环境导致故障回滚或需要紧急补丁的变更比例），在历史走势中通常处于平稳波动区间。然而引入 AI 工具后，该指标的振幅被显著拉大，展现出极端的不稳定性。这表明 AI 既能协助优秀工程师写出更具防御性的健壮代码，也会在缺乏经验或盲目信任的情境下，以极高频率向主干分支倾倒隐蔽的逻辑缺陷，放大系统的脆弱性。
与此同时，代码库正在面临严峻的结构性挑战，集中体现在**代码膨胀**（Code bloat: 系统中出现大量冗余、过度设计或结构松散的代码行数）与**代码流失率**（Code churn: 新提交的代码在极短周期内被频繁修改或推倒重写的比例）的双重恶化：
* **维护认知负荷激增**：尽管调查中开发者对代码可维护性的主观信心提升了近 4%，但微观代码审查揭示出系统中的边缘分支、冗余逻辑和非必要的模板代码正在野蛮生长。
* **单位代码的潜在缺陷率上升**：行业铁律始终成立——系统中的每一行新增代码，在本质上都是潜在的故障载体与维护债务。AI 使得复制粘贴与生成样板代码的边际成本骤降为零，直接诱发了代码行数的无序膨胀。
* **破坏增量交付的工程纪律**：现代高绩效工程组织的核心原则是**小步快跑与增量交付**（Incremental delivery），确保每个变更集可读、易测、易回滚。而 AI 驱动的大规模巨型 PR，严重破坏了审查者的心智模型，不仅阻碍了回滚效率，更导致代码上线后短期内因缺陷暴露而引发大量紧急返工重构。
Original English Source
Here's what our data shows: look at this curve. It's like a roller coaster. When we look at the rate of failed changes in our platform data, we see significant instability. Overall, the trend remains the same, but the amplitude has changed due to the influence of AI. We always see these fluctuations in volatility, but usually not to such extremes, and this is exactly the effect of AI that we are observing. Here is a very interesting point. These are two quality metrics that are usually synchronized with each other. Code maintainability: How easy do developers find the code to modify? According to our data, this figure increased by almost 4%. By the way, in this study we analyzed about 400 organizations, and over the course of the year, we surveyed more than 5,000 developers. So we have pretty solid data. But code churn—this is the percentage of code that changes within three weeks of being committed. That is, if code is committed and changed again within two to three weeks, it is code churn. And this code churn has increased over the year. What does this mean? It means code is being changed more often. It also means code is being written faster, but it is being redone more often. In other words, developers feel more confident, but actually, they are doing more work to rewrite that code. Why is that? There are a few reasons. One of them is that PR size is increasing, which means code is not being delivered incrementally. That's one of the reasons. Another reason is that the total amount of code in the organization is simply increasing. People are adding more and more code, creating code bloat, because it's easier to create code, right? So, there are many reasons, but it's worth emphasizing that every additional line of code is a potential bug. This is a very important fact that we all know, but we forget. And people write more code without thinking about this. So, code bloat is a real problem. Because we also see that this is related to the perception of incremental delivery. Incremental delivery is decreasing. Developers are not working incrementally as much as they used to. They write large blocks of code, and that code is harder to understand, harder to review, harder to test. And incremental delivery, as we've learned, is very important to the way we try to work, so that changes can be easily rolled back, and so that we can understand and test the changes that have been implemented.
---
### 经验分层与落地鸿沟：初级开发者断层与大厂困局
在团队梯队的人群分布上，AI 对不同资历开发者的赋能表现呈现出反直觉的对比。数据显示，**初级工程师**（Junior developers）对 AI 工具的使用频率和依赖度高居榜首。回顾从汇编、面向对象革命到互联网浪潮的历史周期，每一次范式跃迁都会促使年轻一代率先拥抱新抽象工具。初级开发者将 AI 视为实时解答疑问、消除冷启动焦虑的交互式加速器；然而，资深工程师由于掌握着系统的深层业务上下文、隐性架构约束与边界故障经验，对黑盒生成的结果保持着高度审慎，其单点效率提升并不像外界预期的那样显著。
这种资历反差在大型跨国软件企业中演变为了普遍的治理困局，暴露出严重的工具与工程断层：
1. **技能空心化与评估失真**：初级开发者若过度依赖自动补全，极易跳过“设计-试错-排障”的深度思维锤炼，无法培养辨别劣质架构与安全隐患的工程直觉。
2. **审查成本单向转嫁**：初级工程师借助生成工具成倍提交代码，但由于缺乏全局架构视野，这些 PR 充斥着伪优雅代码与边界漏洞，导致资深工程师不得不将大量高价值时间消耗在冗长的代码审查中。
3. **企业级规模化落地阻滞**：诸多上万开发人员体量的一线科技巨头，在斥资千万全员部署 GitHub Copilot 等工具后，并未在组织级 DORA 指标中捕捉到预期的跃迁。这促使管理层从“人手一把枪”的狂热，转向更加理性的体系化反思。
Original English Source
OK. What about demographic differences? What is the difference in the impact of AI on, you know, junior developers and others? Junior engineers use AI the most. This should not surprise anyone. Every time there is a certain leap, and I've been writing code professionally since the late 90s, I'm a bit " old school" and I go through these phases where you adapt and see changes in how work is done, and junior developers adapt first. They use it the most, they benefit from it the most, they feel the biggest increase in productivity. And that makes sense. Because if you're a junior engineer, you don't know what you don't know, and you can ask AI questions you'd be embarrassed to ask a colleague. You can get help without having to ask someone else. Senior developers, on the other hand, show less productivity growth. Why? Because they already know how to write code. They are already productive. And so the increase in their speed is not as significant. But here's what's interesting: when we talk to our enterprise clients—these are large companies with thousands, tens of thousands of developers—they are all trying to figure out how to scale this. They give everyone Copilot, right? Everyone gets Copilot. But then they look at their metrics and they don't see the impact they expected. They say: "We spent all this money on licenses, but we don't see a 50% increase in speed, we don't see an improvement in quality. What's going on?" And the reason is that it's not enough to just give people tools. You have to change how work is done. You have to change the culture, you have to change the processes.
---
### 度量架构与工程重构：从单点度量到上下文工程
为了打破工具部署与商业价值脱节的僵局，企业必须在工程治理层面完成从“孤立点状度量”向“系统化因果分析体系”的升维。DX 提出的 **AI 度量系统**（AI measurement system: 将底层 API 遥测、工具交互频率与交付指标进行多变量关联分析的度量架构）主张：切忌孤立查看工具的使用率或补全采纳率，必须将具体的 AI 使用群体与组织长期的 DORA 基线和质量看板进行跨维度交叉比对。唯有建立起包含**前置感知、系统质量与组织交付**的多层因果闭环，管理层才能穿透虚高的活跃度数据，评估技术对端到端业务流速与系统稳定性的真实映射。
更深层次的转变在于从底层重塑软件架构的设计范式，全面拥抱**上下文工程**（Context engineering: 面向 AI 代理与大模型，对架构契约、数据模式和领域知识进行结构化治理的工程学科）。过去数十年遗留的代码债务不仅阻碍人类协作，更成为 AI 理解系统的致命路障；由于缺乏足够的结构化上下文，AI 代理在面对模糊指令时极易陷入幻觉与试错泥潭。因此，现代软件工程必须推行一系列底层重构策略：
* **确立显式强契约接口**：全面推行具备严密类型定义的 **强类型 Schema**（Strict schemas: 如 Protobuf、OpenAPI 或强类型语言模型对象），消除隐式传参和模糊结构。
* **构建语义自解释系统**：编写结构化、具备强指引性的高质量技术文档与语义元数据，将其作为系统的一等公民纳入工程资产。
* **打通代理端到端反馈闭环**：主动采集并分析 AI 代理在执行任务时遭遇的上下文缺失警报、人工介入干预点以及纠错轮次（Feedback loops），将下游阻滞反哺为上游系统设计的优化输入。
Original English Source
So, how do we measure this? How do we understand what's really happening? This is where we need to look at these proven metrics that we already use. For example, we might want to analyze user groups. We might want to take data from API telemetry and tool performance to understand who is using what and where, but we would like to compare these user groups against our baseline metrics. How does this actually affect quality? How does this actually affect speed? How does this affect the organization and value creation? This is where our AI measurement system comes from . I won't spend too much time on this today, but it is a systematic approach to measuring the impact of AI, which combines developer sentiment, objective speed metrics, and quality metrics into a single coherent system. And this brings us to what we call "Context Engineering." What is Context Engineering? When we talk about AI, we talk a lot about models, right? We talk about OpenAI, Anthropic, Gemini, which model is better, which has a larger context window. But the reality is that the model is only part of the equation. The other part, the more important part, is the context you provide to the model. How well do your systems describe what they do? How good is your documentation? How clear are your APIs? What we're finding is that organizations that have invested in good engineering practices, in good documentation, in clear domain models, are getting much more value from AI than organizations that have messy, undocumented codebases. And this is great news, because it means that all the things we've been saying we should do for years—clean code, good architecture, thorough documentation—are now even more important. Now we have an economic reason to do it, because it directly impacts how well AI can assist us. So, these are things like clear, precise, and well- structured documentation, data structures with strict schemas, and clean architectural boundaries. If you give an AI agent a codebase that is a tangled ball of mud, it will produce more tangled mud. If you give it a well-structured system with clear contracts, it can actually help you extend and maintain that system. And finally, we can finally start making the investments that we should have been making for the past decades. We also want to understand the effectiveness of how we work with AI. And this is one way to do it. What you see here is feedback, quality feedback from agents telling us where they encountered human-driven issues, the context they were given, and the feedback loops they had to go through. So we actually get this data from agents to try to figure out how effectively our teams are working with AI. Are they giving them the right context? Are the tasks well-defined? Or are they just throwing vague prompts at the model and hoping for the best?
---
### 先锋实践与范式转移：全链路 Agent 编排与净价值创造
跨越效能鸿沟的标杆团队，早已超越了简单的代码补全工具依赖，转而进入全流程的 **AI 代理编排**（Agent orchestration: 协调多个具备特定角色与工具调用能力的智能体端到端执行研发任务的模式）。以一家处于行业前沿的先锋团队为例，该团队在保持业务规模扩张的同时不仅没有裁员，反而在单工程师价值产出提升约 15% 的驱动下，创纪录地扩大了工程团队招聘规模。他们通过深度整合 AI 代理与底层工程管线，实现了一整套革命性的研发生态：
* **门禁式自动化代码审查**：构建自动化 Agent 审查管道，每周对 Pull Request 触发超过 3,000 次深度审查，自动拦截浅层逻辑错误、类型偏差与架构契约破坏，将人类审查者彻底解放出来专注于核心业务设计。
* **非同步敏捷协作**：利用 Agent 自动汇总任务进展、代码状态与关联上下文，将传统的每日站会精简至每周仅需两次，消除了跨时区协同的碎片化心智损耗。
* **知识内化与秒级应急响应**：为 **站点可靠性工程**（Site Reliability Engineering, SRE: 运用软件工程方法保障生产系统高可用与弹性的学科）配备专属故障排查 Agent。当线上告警触发时，Agent 自动聚合历史故障手册、系统日志与链路上下文并推送至通信群，使响应者免于漫长低效的排障深潜，将工程师入职（Onboarding）适应期压缩至创纪录的两周。
这一系列实证数据表明，软件开发的未来绝非由盲目追求代码堆砌的孤立工具所主导。真正的效能飞跃，源于将高质量的领域工程资产、严谨的系统度量框架与嵌入研发生命周期的智能体协同进行深度融合，最终完成从局部行级自动补全到组织级净价值创造的质变。
Original English Source
Despite the additional growth in PR throughput, our median growth in the study we conducted from November to February was about 15%. This is a good number, right? 15% increase in PR throughput is significant. But it's not 50%, and it comes with the quality tradeoffs we discussed. Now I want to share a story about a team that is doing this really well. This is a real company, a forward-thinking organization that took a different approach. Instead of just giving developers a tool and hoping for productivity, they redesigned their entire workflow around AI agents. What did they do? First, they automated code reviews. They automate about 3,000 code reviews per week, triggered by pull requests, and they check superficial things, compliance, style, basic logic, so that when a human reviewer looks at the code, they can focus on high-level architecture, business logic, and security. Second, they restructured their team communication. Instead of endless status meetings, daily stand-ups, etc. Using agent submissions and similar tools in the ecosystem they created , they reduced the number of stand-ups to two times a week instead of five . They successfully onboard engineers in about two weeks, which is very fast. Typically, the industry standard for this process is over a month. But what I like most about this story is that they see an additional value creation of about 15% per engineer. So they are hiring more people than ever before in the company's history because they are getting more value from each person they bring on board. They created an agent for SRE that collects troubleshooting steps from instructions, information, and context about the incident, and then integrates it all into SRE communication channels. So when an incident occurs, the SRE immediately gets context and doesn't waste minutes searching for a solution to the problem. So, if you want to dig deeper into this data, our Q2 report will be out in a few weeks, but the Q1 report still has a lot of interesting things from this session . And if you subscribe to our newsletter or return to the site sometime in mid- to late July, we will already have a report for the second quarter. And if you want to see a preview of this report, come to my talk this afternoon. Thank you all for your time. I appreciate it.