Hello everyone. Today, we'll be talking about how to build coding agents. I'm Bill, and I work on the applied AI startups team at OpenAI. Brian and I work together on the OpenAI startups team, and we specifically focus on building coding agents here.

So, why are we giving this talk about coding agents? It's quite interesting because this field has been booming for the past year. If you think about it, it hasn't been that long—only about a year or so. The landscape for coding agents, especially concerning the harness, is constantly shifting. What makes it interesting is that it's a signal of how close we are to **AGI** (Artificial General Intelligence: 能够理解、学习或执行任何人类智力任务的假设智能). Software engineering can be seen as a universal medium for problem-solving. However, because the field is evolving so rapidly, we've had to constantly rebuild the agent on top of the model whenever a new model is released. Today, we're going to discuss how we might overcome this challenge.

Here's what we'll cover today. We'll start with the anatomy of a coding agent, delving into the details of models and harnesses and their interplay. We'll share lessons learned from our own development. Specifically, we'll discuss **Codeex** (OpenAI's coding agent: 一个由OpenAI开发的编码智能体，结合了模型和协调器，旨在帮助开发者更高效地编写代码), our own coding agent. We'll also explore emerging patterns for using agents like Codeex in your products. Finally, we'll discuss what to expect from Codeex in the future, so you can build alongside us.

To begin, let's discuss what constitutes a coding agent. It's quite simple, though people tend to overcomplicate it. It's made of three parts: a user interface, a model, and a harness. The interface is self-explanatory; it could be a **CLI** (Command Line Interface: 命令行界面，通过文本命令与计算机交互的界面) tool, an **IDE** (Integrated Development Environment: 集成开发环境，提供代码编辑、编译、调试等功能的软件), or a cloud/background agent. Models are also straightforward, like the latest **GPT-5.1** (Generative Pre-trained Transformer 5.1: OpenAI开发的大型语言模型系列之一) series, Codeex Max, or models from other providers. The harness is the more interesting part. It directly interacts with the model, essentially a collection of prompts and tools combined in a core agent loop that manages input and output. This last part will be our focus today.

As mentioned, coding is one of the most active frontiers in applied AI. With models constantly being released, the challenge for everyone is the need to continuously adapt agents to new models.

Bill has given us a great overview of coding agents and their components. Let's zoom in on the harness, which turns out to be quite tricky. A harness is the interface layer to the model. It's the surface area through which the model interacts with users and code, and performs actions using tools. It comprises all the elements the model needs to operate over multiple turns, call tools, write code, and interpret user requests. For some, the harness might be the product's 'special sauce,' but building a good one is genuinely challenging. We'll discuss how we achieved this.

Let's look at some of these challenges. For instance, your brand-new, innovative custom tool for your agent might not be something the model is accustomed to using; it may never have seen it during training. Even if it has, you'll need to spend time tuning your **prompt** (提示: 引导AI模型生成特定输出的指令或问题) to that specific model and its inherent habits. New models are constantly emerging. Then there's **latency** (延迟: 系统响应请求所需的时间): Does the model take a while to process certain things? How do you prompt it to avoid unnecessary thinking? How do you present the user experience of a thinking model while it's processing? Does it communicate its thoughts, or do you need to summarize them? Managing the **context window** (上下文窗口: AI模型在单次交互中能够处理的输入和输出文本的最大长度) and **compaction** (压缩: 优化上下文窗口内信息以提高效率和减少成本的技术) can be extremely challenging. We just launched Codeex Max, which handles this out-of-the-box, freeing you from worrying about compaction and context window management, as it's genuinely difficult to do yourself. Furthermore, **APIs** (Application Programming Interfaces: 应用程序编程接口，允许不同软件组件相互通信的规范) are constantly changing, with completions, responses, and future functionalities. How does the model know how to leverage these to extract the most intelligence inherently?

This is the interesting part: fitting a model into a harness requires extensive **prompting** (提示工程: 设计和优化给AI模型的指令，以获得期望的输出). It turns out that a model's training methodology has side effects. I view it as 'intelligence plus habit.' Intelligence refers to what the model excels at—which languages it knows well, its code-writing capabilities in specific frameworks. Habits refer to the learned approaches it uses to solve problems. We've trained our models to plan solutions, gather context, think before coding, and then test their work. Developing an intuition for these habits is key to becoming a good **prompt engineer** (提示工程师: 专门设计和优化AI模型提示以提高其性能和输出质量的专业人员). If you don't instruct the model in familiar ways, you'll encounter issues. We observed this with GPT-5's launch. Many users, unfamiliar with our coding models, tried to adapt prompts from other models for GPT-5, but our model was trained with different inherent behaviors. For example, when prompted to meticulously examine every file before a code edit, our model was overly thorough, leading to long processing times and suboptimal performance. We discovered that by allowing the model to follow its accustomed behaviors and avoiding over-prompting, it performs significantly better. We even asked the model directly, 'I like the solution, but it took a long time. What can I do differently in your instructions to help you get there faster?' It literally responded, 'You're telling me to look at everything, which I don't really need to do, and that's what's taking forever.'

Thus, you can see the advantages of building both the model and the harness together, as you inherently understand all these nuances during development. That's why Codeex is a combined model and harness.

Let's delve deeper into Codeex and its capabilities. We built Codeex to be an agent for every coding environment. It's a **VS Code** (Visual Studio Code: 微软开发的一款免费开源的代码编辑器) plugin, a CLI tool, and can be invoked in the cloud from VS Code or even ChatGPT on your phone. Fundamentally, you can use it to transform your specifications into runnable code, starting from a prompt. It can formulate a plan, navigate your repository to edit files, run commands, and execute tasks. You can call it from Slack or have it review **PRs** (Pull Requests: 在软件开发中，开发者提交代码更改以供审查和合并到主代码库的请求) on GitHub. All these expected functionalities mean Codeex's harness must handle highly complex tasks. A Codeex team member emphasized that it's far harder than it seems. You must manage parallel tool calls, including thread merging. Consider security aspects like **sandboxing** (沙盒: 一种安全机制，将程序隔离在受限环境中运行，以防止其对系统造成损害), prompt forwarding, permissions, and port management. Compaction is another significant challenge; doing it well is complex. When do you trigger compaction? When do you reinject? How do you optimize cache during **MCP** (Multi-Context Processing: 多上下文处理，指AI模型在处理任务时管理和整合多个独立上下文的能力)? All the underlying infrastructure for MCP support must be built into the harness. Not to mention image handling—determining the optimal resolution for compression before sending them to the model. All this work is required if you build from scratch and need to keep it updated as new features emerge.

We've bundled all these features into an agent that can safely write its own tools to solve new problems it encounters. This means we essentially have a computer use agent for the terminal. That sounds much more powerful than just a plain coding agent, doesn't it? But consider this: before browsers and graphical user interfaces existed, wasn't that how we always operated computers—by writing code and chaining commands together in a command-line interface? So, if you can express your tasks as command-line operations or file tasks, Codeex will know what to do. For example, I use Codeex to organize many photos from my desktop into a folder—a simple use case. But it can also analyze vast amounts of CSV files within a folder, performing data analysis. It doesn't have to be a coding task; if it can be accomplished by running tools from the command line, you can use Codeex.

Now that we see Codeex is such a powerful harness, I want to share how you can use it to build your own agents. You can essentially use the Codeex agent *inside* your own agent. How does this work? If you're building the next coding startup, we don't have all the answers, but we've identified a few patterns from working with top coding customers like Cursor and VS Code that might help. One such pattern is the harness becoming the new **abstraction layer** (抽象层: 隐藏底层复杂性，提供更简洁易用接口的软件层). The benefits are obvious: you no longer need to prioritize optimizing prompts and tools with every model upgrade. Brian asks if this means simply building a wrapper. Bill disagrees, stating that calling it a 'wrapper' is reductive to the infrastructure layer's value proposition. This pattern allows you to focus most of your efforts on differentiating your product, which is where most of the value lies. Let's look at some patterns that have helped our customers. Codeex is an **SDK** (Software Development Kit: 软件开发工具包，包含开发特定应用程序所需的所有工具和库). It can be called via a TypeScript library or a Python `exec`. There's a GitHub Action to automatically merge PR conflicts. You can also add it to an agent's SDK, providing MCP connectors back to your product. So now you have an agent. We started with chatbots you could talk to, then gave them tools. Now, you can give your chatbot a tool that can *make other tools* it doesn't have. This enables building enterprise software that writes its own API-level plugin connectors for each customer on the spot—a task previously handled by professional services teams. You get fully customizable software that can self-interact. For Dev Day, I built a Kanban board that could fix its own bugs, which was quite fun. Lastly, you can emulate what Zed has done: they wrapped Codeex within a layer, providing an interface to their IDE for user interaction and code edits. This frees them from constantly managing the complexities we handle, allowing them to focus on building the best code editor.

Our top coding partners, like GitHub, have used this to great effect. We've created an SDK that they use to directly integrate with Codeex. You can also use this SDK to control Codeex as part of your **CI/CD pipeline** (Continuous Integration/Continuous Delivery: 持续集成/持续交付，一种自动化软件开发流程，旨在提高开发效率和代码质量), or as an agent that directly interacts with your own agent. If you want to customize the agent layer, you can. For instance, we worked closely with the Cursor team to extract the best performance from the Codeex *model* (not the agent—we're bad at naming things; the model is distinct from the agent). They achieved this by aligning their tools with the model's training distribution and by aligning their harness with our open-source implementation of the Codeex CLI. All of this is publicly available. You can fork the repo, use our source code—go nuts!

So what does the future hold for Codeex? It hasn't even been out for a year. Um and especially with the launch of CEX match yesterday like things are really changing fast. Uh it's the fastest growing model in usage now serving dozens of trillions of tokens per week which has actually doubled since dev day.

It's always good to build where the models are going. It's safe to assume that the models will get better. They'll be able to get to work on much longer horizon tasks unsupervised. New models will raise the trust ceiling. I trust these models now to do some way harder work than I would have 6 months ago. And that's going to keep increasing. The future is about sprawling code bases and non-standard libraries and knowing how to work in closed source environments, matching existing templates and practices and the models uh and and and so you can imagine that the SDK will evolve to better support these model capabilities, letting the model learn as it goes and not repeat mistakes and generally provide more surface area for an agent that writes code and uses a terminal to solve whatever problems it encounters. counters and you can use that in your products via the SDK.

So, what have we learned? Harnesses are really complicated and take a lot of work to maintain, especially with all the new models coming out. So, we've built one for you inside of Codeex that you can use off the shelf or look at the source if you want to and you can use it to build new things outside of coding and let us do all of the work making sure that you have the most capable computer agent.

And we're really excited to see what you craft.