<clear Chinese title grounded in the source>

Heat.

Hey, heat. Hey, heat.

Heat. Heat.

Heat.

Hey. Hey.

英文原文

Heat.

Hey, heat. Hey, heat.

Heat. Heat.

Heat.

Hey. Hey.

距离今天的议程开始还有10分钟。

英文原文

There

are

10

minutes

remaining

until

today's

programming

begins.

Heat. Heat.

I feel deep.

英文原文

Heat. Heat.

I

feel

deep.

[music]

英文原文

>> [music]

Heat. Heat. Heat.

Down.

Down.

feel.

Come on.

英文原文

>> Heat. Heat. Heat.

Down.

Down.

feel.

Come

on.

大家早上好，欢迎回到2026年AI Engineer Code Summit的最后一天。过去几天是一段令人难以置信的学习和交流想法的历程。今天我们将继续保持这一势头，共同探索AI工程的下一步发展，期待接下来又一个令人兴奋的一天。为了开启我们最后一个上午的主题演讲，请大家热烈欢迎我们的主持人——DuckBill Group的首席云经济学家Cory Quinn。

英文原文

Good

Good

morning

and

welcome

back

to

the

final

day

of

AI

Engineer

Code

Summit

It's

been

an

incredible

few

days

of

learning

and

sharing

ideas.

Today

we're

keeping

that

momentum

going

as

we

explore

what's

next

for

AI

engineering

and

look

forward

to

another

exciting

day

ahead.

To

kick

off

our

final

morning

of

keynotes,

please

put

your

hands

together

for

our

MC

Cory

Quinn,

chief

cloud

economist

at

the

Duck

Bill

Group.

主持人致辞

早上好。今天早上这里光线有点晃眼。呃，希望大家昨天都过得很开心。从那时起我们有没有学到一些令人兴奋的新东西？我的意思是，如果我们看了新闻就会知道，无论我们今天讨论什么，好坏不论，我们都不太可能成为今天的AI头条新闻。呃，在过去的这两天里，我们进行了一系列极其精彩的研讨会、主题演讲、交流对话和各种展览环节。有人告诉我，昨天本次大会的在线直播创下了新纪录，呃，有超过5万人观看了直播。所以，不仅仅是这个房间里的人，还有坐在家里的各位。呃，我想象中大家在不同意台上人的观点时，会对着电脑屏幕大喊大叫——这比坐在第一排直接喊稍微礼貌一点，但你知道，这种情况总是会发生的。

英文原文

Good

morning.

Little

bright

this

morning

here.

Uh,

hope

everyone

had

fun

yesterday.

Did

we

learn

new

and

exciting

things

since

then?

I

mean,

if

we've

read

the

news,

we

know

that

whatever

we

talk

about

today,

it's

unlikely

we're

going

to

be

today's

AI

top

story

for

better

or

worse.

Uh,

we've

had

over

the

past

two

days

a

tremendous

run

of

workshops,

keynotes,

conversations,

and

various

expo

sessions.

I'm

told

that

yesterday

a

new

high

score

on

the

live

stream

for

this

conference,

uh,

over

50,000

people

have

watched

it.

So,

it's

not

just

the

people

in

this

room,

it's

the

folks

at

home.

uh

I

assume

in

my

imagination

yelling

at

their

computer

screen

when

they

disagree

with

people

slightly

more

polite

than

doing

it

from

the

front

row,

but

you

know

that

happens.

嗯，所以今天当我们进入这里的最后一天议程时，日程排得很满，充满了大量令人兴奋的对话、有趣的人物以及一些很棒的演讲。每当你听到有人说“哦，这里有一些很棒的演讲”时，我心里总会默默补上一句：“还有三个讲得很烂的”。呃，我觉得目前这种情况还不会发生，不过我们大家都会一起见分晓的。不要有压力。呃，在我们正式开始之前，我想再次感谢让这样一场活动成为可能的合作伙伴和赞助商。呃，首先当然是Google DeepMind，据各方评价，他们最近表现十分抢眼。为他们感到高兴。我们还有我们的休息区赞助商。是的，有我们的休息区赞助商和白金赞助商，呃，同样是他们让休息区以及诸如此类的其他便利设施成为可能。还有金牌赞助商。呃，再次说明，没有大家在这些事情上的帮助，我们是办不成这次大会的。我们的银牌赞助商也是类似的情况，只是在致谢顺序上稍靠后一点。当然，还有所有的支持合作伙伴，同样是所有这些公司在没有义务的情况下共同出力，促成了这样的活动。没有帮助你是办不成的。我的意思是，咖啡不会自己摆好、自己收走、然后再自己摆好、自己收走。这需要人力。

英文原文

Um

so

today

as

we

head

into

our

final

day

of

programming

here,

it's

a

full

schedule

full

of

a

bunch

of

exciting

conversations,

interesting

people,

and

some

great

talks.

U

whenever

you

hear

someone

say,

"Oh,

there

are

some

great

talks."

I

always

mentally

finish

it

with

and

three

terrible

ones.

Uh,

I

don't

think

that's

going

to

happen

yet,

but

we'll

all

find

out

together.

No

pressure.

Uh,

before

we

dive

in,

I

do

want

to

thank

once

again

the

partners

and

sponsors

who

make

an

event

like

this

possible.

Uh,

obviously

starting

off

with

Google

DeepMind

who

has

been

uh,

cooking

with

gas

lately

by

all

accounts.

Good

for

them.

We

also

have

our

lounge.

Yeah,

there

are

our

lounge

and

platinum

sponsors

uh,

who

again

make

the

lounge

and

other

nonsense

like

it

help

possible.

the

gold

sponsors

as

well.

Uh

again,

we

couldn't

do

this

without

folks

helping

on

these

things.

Our

silver

sponsors,

similar

boat,

just

a

little

bit

later

in

the

sequence.

And

of

course,

all

of

the

supporting

partners

who

again,

collectively,

all

these

companies

pitch

in

when

they

don't

have

to

to

help

something

like

this

happen.

You

can't

do

it

without

help.

I

mean,

the

coffee

doesn't

set

itself

up

and

take

itself

away

and

set

itself

up

and

take

itself

away

all

by

itself.

It

takes

people.

呃，所以这基本上就是我们目前的情况。这是一场令人振奋的盛会。我们期待着圆满收官。但今天的一个重要主题是雄心（Ambition）。有人可能会说，在过去三年里，这一直是整场AI革命的主题。但在这里，它是关于把格局想得再大一点。为了做到这一点，我们今天的开场演讲题目是《AI中的雄心》（Ambition in AI）。呃，Jason Louu在OpenAI负责开发者体验，我很期待看到他要为我们分享什么。让我们热烈欢迎他。Jason。

[applause and cheering]

英文原文

Uh

so

that's

effectively

where

we

are

now.

It's

been

an

exciting

event.

We

are

looking

forward

to

bringing

it

home.

But

one

of

the

big

themes

of

today

is

ambition.

One

could

argue

that's

been

a

theme

of

the

whole

AI

revolution

for

the

last

three

years.

But

here

it's

it's

about

thinking

a

little

bit

bigger.

And

to

do

that

we're

our

opening

talk

today

is

titled

ambition

in

AI.

Uh

Jason

Louu

is

in

developer

experience

over

at

Open

AI

and

I'm

excited

to

see

what

he

has

to

say

for

us.

Let's

please

give

him

a

warm

welcome.

Jason

[applause and cheering]

AI中的雄心：提升愿景与责任

大家好，我先拿一下翻页笔。

太好了。我们先看看这些幻灯片能不能显示出来。

好的。大家好，我是来自OpenAI开发者体验团队的Jason，我非常兴奋能为大家开启这个关于提升雄心的演讲板块。

英文原文

Hey

everyone,

let

me

just

get

the

uh

clicker

real

quick.

Sweet.

Let's

see

if

these

slides

show

up

first.

All

right.

Hey

everyone,

I'm

Jason

from

the

developer

experience

team

at

OpenAI

and

I'm

super

excited

to

kick

off

this

track

on

raising

your

ambitions.

你知道，我是三月份加入的，有那么几周时间Codex应用其实还算不上成型。那时也还没有Astra。当这些东西推出时，我有点意识到，自己不再是一个仅仅在开发者体验团队工作的人，由于AI现在可以成为一种让所有人都能创造事物的工具，我是在“全方位体验团队”（everything experience team）工作。因此，我希望这次演讲不仅能面向在座的各位开发者，也能面向线上的所有人。大家知道，在这一年里，我们做了很多不同的事情，对吧？我们想要一个更好的编程应用，于是我们构建了Codex应用。我们一直在为生物学等领域做出贡献。大家一直想要制作个性化应用程序，所以我们推出了Sites、网络安全、语音、计算机操作。在过去的几个月里，我们做了一些非常雄心勃勃的事情，比如我们在数学和物理学方面的贡献，以及许多新的物理工程能力。嗯，想想挺疯狂的，几年前我们还在做小学数学，而现在我们能够去攻克纳维尔-斯托克斯方程（Navier-Stokes）。

英文原文

You

know,

I

joined

in

March

and

the

Codex

app

wasn't

really

a

thing

uh

for

a

couple

weeks.

There

was

no

Astra

yet.

And

when

these

things

launched,

I

kind

of

realized

that

I

stopped

becoming

someone

that

worked

on

the

developer

experience

team

and

I

worked

on

the

everything

experience

team

now

that

AI

can

be

a

tool

that

allows

everyone

to

make

things.

And

so

I

want

this

talk

to

not

only

to

be

addressable

to

the

people

here

in

this

room

as

developers,

but

for

everyone

online

as

well.

You

know,

in

this

year,

we've

done

a

lot

of

different

things,

right?

We

wanted

a

better

coding

app.

We

built

a

Codex

app.

We

've

been

contributing

to

things

like

biology.

Folks

have

wanted

to

make

personal

applications.

So,

we

launched

sites,

cyber

security,

voice,

computer

use.

And

in

this

past

couple

of

months,

we've

done

some

really

ambitious

things

like

our

contributions

to

math

and

physics

and

a

lot

of

new

physical

engineering

capabilities.

Um,

it's

kind

of

crazy

to

think

that

a

couple

years

ago,

we're

doing

great

school

math

and

now

we're

be

able

to

like

tackle

Navia

Stokes.

所以当有人让我多谈谈雄心意味着什么时，我认为我反复回想的问题是：雄心其实并不在于你怎么做事情，而在于你怎么看待这些事情，对吧？你想做出什么，以及在思考你所做的事物时你有多大的抱负？

我知道这终究是一场编程大会，这里的许多演讲都会讨论诸如如何运行更长时间的任务，或者我想会讨论黑灯工厂（dark factories）和多智能体，嗯，但我想对于像我这样的人来说，仅仅说我有数千个智能体在后台运行，可能并没有那么容易引起共鸣，对吧？我有无限的token。因此，我真正想做的是进行一场演讲，多谈谈我认为未来开发者的角色和责任是什么，对吧？所以，使用更多的token并不是我所说的更有抱负。在我看来，在如今执行已经变得免费的世界里，我认为雄心可以概括为以下这三项原则，对吧？你是否真的相信这项工作是你创造的，并且你对这项工作拥有归属感（ownership）？你是否能描述出你的品味是什么，并理解你该如何指示模型把事情做得更好？你是否能够描述出什么是好的标准？最后，你是否真的有足够的好奇心去使用AI追求其他领域，对吧？你是否真的能够拓宽你的视野，而不是总想着自己会被取代之类的？

英文原文

So

when

I

got

asked

them

to

talk

a

little

bit

more

about

what

ambition

meant

I

think

the

question

I

keep

going

back

to

is

like

ambition

isn't

sort

of

how

you

did

do

things

but

like

how

you

approach

these

things

right

what

do

you

want

to

make

and

how

ambitious

are

you

and

thinking

about

the

things

that

you

make

I

know

this

is

a

coding

conference

after

all

and

many

of

the

talks

here

will

talk

about

things

like

how

to

run

like

longer

running

tasks

or

I

think

talk

about

like

dark

factories

and

multi-

aents

um

but

I

think

from

someone

like

me

like

it

might

not

be

as

relatable

to

just

say

that

I

have

like

thousands

of

agents

running

in

the

background,

right?

I

have

unlimited

tokens.

And

so

really,

I

want

to

give

a

talk

that

talks

a

little

bit

more

about

like

what

I

think

is

the

role

and

responsibility

for

developers

in

the

future,

right?

And

so

like

using

more

tokens

isn't

what

I

mean

by

being

more

ambitious.

To

me,

in

a

world

where

execution

is

now

free,

I

think

ambition

is

summed

up

by

these

three

principles,

right?

Do

you

actually

believe

that

you've

created

this

work

and

that

you

have

ownership

of

this

work?

Can

you

describe

what

your

tastes

are

and

understand

how

you

can

tell

the

model

to

make

things

better?

And

can

you

describe

what

good

looks

like?

And

then

lastly,

are

you

actually

curious

enough

to

use

AI

to

pursue

other

fields,

right?

Are

you

actually

able

to

expand

your

horizon

rather

than

thinking

that

you're

going

to

be

replaced

or

something

like

that?

而这主要源于人们在更多使用AI时所产生的三个不同的担忧：我们是否只是在给这个世界制造更多的粗制滥造的内容（slop）？AI是否会摧毁我批判性思考的能力？以及AI是否会取代我们，对吧？因此，我想逐一探讨这些问题，并以此作为构建这次关于雄心的对话的框架。

所以首先，你真的要考虑对自己交付的工作拥有归属感。因此第一个问题是，我们是否会交付更多粗制滥造的内容，对吧？答案很可能是肯定的。但我尝试以我们当初看待照相机刚问世时的方式来思考这个问题，对吧？借此举手示意一下，在座有多少人觉得他们所做的技术工作是一种创造性的追求？

英文原文

And

this

mostly

comes

from

three

different

fears

that

people

have

when

it

comes

to

like

using

AI

more.

Are

we

just

going

to

create

more

slop

in

the

world?

you

know,

is

AI

going

to

destroy

my

ability

to

think

critically

and

is

AI

going

to

replace

us,

right?

And

so

I

want

to

talk

about

these

things

one

at

a

time

and

use

that

as

a

way

of

framing

this

conversation

about

ambition.

And

so

the

first

thing

is

you

really

want

to

think

about

owning

the

work

that

you

ship.

So

the

first

question

is

like

are

we

going to

ship

more

slop,

right?

And

probably

the

answer

is

yes.

But

I

try

to

think

about

this

the

same

way

that

we

thought

about

the

camera

when

it

first

came

out,

right?

as

a

way

of

raise

of

like

show

of

hands

like

how

many

people

here

feel

like

the

technical

work

they

do

is

a

creative

pursuit.

是的，我也是。呃，这些天来我不再认为自己是一名开发者。我多半认为自己是一个试图将事物凭空创造出来（will things into existence）的人，对吧？我必须再次体会到这种归属感，并描述出我喜欢什么。在过去的生活中，没多少人知道这一点，我曾经是一名机器学习工程师。我曾经是一名物理学家，也曾经是一名艺术家。随着我们引入更多实现这种复制的方法，对吧，作为这种创造事物的方式，照相机的故事是我不断在思考的事情。

当照相机刚问世时，那些忙于专注于自己重现图像能力的画家们，是的，他们确实感到有些焦虑。但在那时，人们甚至没有去想摄影是否能够改变艺术界。我们没有去想一部电影意味着什么。当你看到一部查理·卓别林的电影刚上映时，很容易把它看作是某种很傻的东西。

但到了某个时刻，你意识到摄影不仅是一种艺术，而且它改变了艺术界，对吧？现在我们有了电影和所有这些其他媒介，有许多惊艳的事物可供你探索。瓦尔特·本雅明在一本书——《机械复制时代的艺术作品》中谈到了这一点，这本书恰好是在照相机流行时出版的。他说：“摄影将手从最重要的功能中解放出来，并将它们转移到了眼睛和声音上，对吧？

英文原文

Yeah,

you

know

me

too.

Uh

these

days

I

don't

think

of

myself

as

a

developer.

I

mostly

am

thinking

about

myself

as

someone

who's

trying

to

will

things

into

existence,

right?

And

I

have

to

again

feel

this

ownership

and

describe

what

I

like.

In

a

past

life,

not

many

people

know

this,

I

used

to

be

a

machine

learning

engineer.

I

used

to

be

a

physicist

and

I

used

to

be

an

artist.

And

the

the

story

of

the

camera

is

something

I

keep

thinking

about

as

we

introduce

more

ways

of

doing

this

kind

of

replication,

right?

As

as

this

way

of

creating

things.

When

the

camera

first

came

out,

painters

who

were

busy

focusing

on

their

ability

to

recreate

images,

yeah,

they

were

a

little

stressed

out.

But

at

that

time,

people

weren't

even

thinking

about

whether

or

not

photography

could

change

the

art

world.

We

weren't

thinking

about

what

a

movie

meant.

You

see

like

a

Charlie

Chaplan

film

when

it

comes

out,

it's

really

easy

to

dismiss

that

as

some,

you

know,

silly

thing.

But

at

some

point

you

realize

that

not

only

is

photography

art,

but

it

has

changed

the

art

world,

right?

And

now

we

have

film

and

all

these

other

mediums

and

there's

amazing

things

that

you

can

explore.

Walter

Benjamin

says

this

in

a

book,

The

Work

of

Art

in

the

Age

of

Mechanical

Reproduction,

a

book

that

came

out,

right,

when

the

camera

became

popular.

He

says,

"Photography

freed

the

hand

of

the

most

important

functions

and

brought

them

to

the

eyes

and

to

the

voice,

right?

现在用 AI 也有点类似，只是我可以在对话中直接使用语音模式。我就直接说，你知道的，做这个、做那个。同样地，在某种程度上，如果你是一名摄影师或导演，你也可以直接说，你知道的，把那台摄像机移到那边去。我想要一个更广的镜头。我想要捕捉某种感觉。也许我身边有一个团队来帮我实现那个构想。

英文原文

This is kind of the same with like AI now except I just get to use voice mode on on chat. I can just say, you know, do this and do that. In the same way that at some point if you're a photographer or director, you can just say, you know, move that camera over there. I want a wider shot. I want to capture some kind of feeling. And maybe I have a team around me that helps me capture that vision.

现在我们生活在一个每个人都拥有相机的世界。确实，也许我相册后台里存着上千张我从来不看的照片。也许那算是个缺陷。但这只关乎我是否记录下了生活中几个真正重要的时刻，对吧？对这些时刻拥有所有权，才是重要的事情。比如，像 30 年前，我其实没有童年照片，因为我们成长的地方根本买不起相机，对吧？现在技术让每个人都能拥有这种途径。如果当时能有一台相机，我肯定会很高兴，哪怕那意味着多出一点废片，对吧？

英文原文

And now we live in a world where everyone has a camera. And sure, maybe I have a thousand photos in my in my background in my my camera roll that I never look at. Maybe that is flaw. But it only matters whether or not I have captured a couple of really important moments in my life, right? And and having ownership over those moments to the thing that's important. Like for example, like 30 years ago, like I didn't really have childhood photos because we just grew up in a place where we couldn't afford a camera, right? And now technology has allowed everyone the ability to have that kind of access. And I sure would have liked to have a camera back then, even if it meant a little bit more slop, right?

所以如果回到画家的这个想法，我认为你必须问自己的问题是，在这个例子里，你认同自己是画匠还是艺术家，对吧？也许这意味着画得逼真并不是真正有价值的技能。但如果你把自己视为艺术家，对吧？不管是画笔还是相机，不管是代码、粘土还是其他任何东西，这些不都只是让你表达自我和创造事物的媒介吗，对吧？我做了一张桌子，我不会说是得伟（DeWalt）做了这张桌子，对吧？就像一位导演哪怕片场有上千人，那部电影依然是导演的电影。即使在开源领域，我们现在可能有成千上万的贡献者，甚至是智能体在运行系统，但为工作质量承担责任的还是核心维护者，对吧？而且我认为现在的危险不在于 AI 是否会制造更多的粗制滥造，对吧？相反，它关乎自我抹杀，对吧？你打算做一个被动观察者看着作品被制造出来并导致这种粗制滥造，还是打算成为创作行为的积极参与者？对吧？我认为很多人都在慢慢疏远这种参与。对吧？一个人提交一千个 PR 和另一个人之间的唯一区别，就在于他们是否真正关心这项工作，是否是一个积极的参与者，你知道吗？

英文原文

And so if you go back to this idea of this painter, I think the question you have to ask yourself is, do you identify yourself as the painter in this example or as the artist, right? Maybe it means that painting photorealistically isn't the skill that's really valuable. But if you think about yourself as the artist, right? Whether it is the paintbrush or the camera, whether it is code or clay or anything else, these are not just mediums that allow you to express yourself and to make things, right? I don't make a table and I say like DeWalt made this table, right? Just because a director has thousands of people on set, it's still the director's movie. And even in open source, we might have thousands of contributors or even agents now running the system, but it's the core maintainers that help help hold the responsibility for the quality of the work, right? And I don't think the danger these days is whether or not like AI is going to create more slop, right? But instead, it's about self-erasure, right? Are you going to be a passive observer of the work that's being created which results in this slop or are you going to be someone who is an active participant in the act of creation? Right? I think many people have been sort of moving themselves away from this. Right? And the only difference between like a person who's making a thousand PRs versus another is whether or not they actually care about the work and whether they are an active participant, you know?

主人翁意识与打磨

你知道，所以作为一名工程师，这意味着你必须更多地去使用自己的产品，注意何时出现问题，并决定是否需要再做一轮优化。你在网上看到的很多精彩案例都不是一蹴而就的，对吧？这些都是你随着时间推移不断迭代出来的东西，因为你为创造这种作品感到自豪。你知道，在内部我们对比如 Codex 应用拥有高度的主人翁意识，产出了很多很棒的成果，对吧？这里的大多数人对产品都有着极强的主人翁意识，因此随着我们交付更多内容，即便有些地方出现疏漏，我们也想把它们聚拢整合好。再次说明，这主要是因为我们真正在乎，对吧？因此，在担心粗制滥造这个问题上，我认为针对粗制滥造的良方，其实纯粹就在于你真的需要坚信你拥有这项工作的所有权。

英文原文

You know, and so as an engineer, what this means is you just have to like use your products more, notice when things break and decide whether things need another pass. Like a lot of the great examples you see online are not oneshot examples, right? These are things that you iterate over time because you feel proud to create this kind of work. You know, internally we have a lot of amazing work with like our ownership over like the Codex app, right? Mo most people here have an incredible sense of ownership over the product and as so as things slip as we ship more, we want to bring them together. And again, this is mostly because of the fact that we care, right? So, in terms of worrying about slop, I think that the the medicine for slop is really just the fact that you really need to believe that you own this work.

我认为要解决的第二点，在于你是否觉得自己能真正描述出好东西是什么样子。为此，你需要走出去，获得灵感。

英文原文

I think the second thing to address is whether or not you feel like you can actually describe what good looks like. And for that, you need to go out and be inspired.

审美品味与语言

所以大家现在问的第二个问题是，你认为 AI 会摧毁我们的批判性思维能力吗？对吧？如果我对工作拥有所有权，我怎么知道某个东西是否足够好？如果在这个类比中我直接递给你一台相机，这是否意味着它剥夺了创作的所有付出？我认为答案是否定的。对吧？相反，现在的关键在于拥有能够描述当下状况的语言。所以当我刚进艺术学院时，你知道，也许我不知道怎么冲洗胶片，我只用我的 iPhone。但我仍然需要理解设计要素以及美是什么样的，对吧？空间、明度、形状、质感。你知道，当时我常常花很多时间去想：你知道吗？我觉得我不理解负空间是什么意思。让我去多画一些草图探索一下吧。

英文原文

So the second question people are asking now it's like do you think AI is going to destroy our ability to think critically? Right? If I own the work, how do I know if something's good enough? If I just hand you a camera in this analogy, like does it mean that it takes away all the effort of creation? I don't think the answer is yes. Right? Instead, it's now about having language to describe what's going on. So when I first started art school, you know, maybe I don't know how to develop film. I just use my iPhone. But I still need to understand like the elements of design and what does beautiful look like, right? Space, value, shape, texture. You know, when I I would just spend a lot of time thinking, you know what? I don't think I understand what negative space means. Let me go do a lot of sketches and explore a little bit.

现在做开发也是一样，对吧？我可以做多个应用，而且我必须思考我为什么喜欢某个东西，为什么不喜欢。我在艺术学院的大部分时间都只是在想，天哪，我可以做东西，我可以看着它们，然后我只会说，我不喜欢它看起来的样子。而且当时我注意到的是，我的品味和判断力超过了我的技能。我做不出我喜欢的东西，对吧？我不能只是看着镜子告诉自己一百遍“让它更有冲击力”，然后就知道它会变好。我必须走出去建立那种语言，对吧？人们说品味就是护城河，但我个人通常很讨厌这句话，因为这根本没告诉你品味究竟意味着什么，对吧？对我来说，品味就是有能力描述你喜欢什么，并预测其他人会喜欢什么，对吧？建立这种关于世界和其他人的模型，那就是新的思考方式，对吧？维特根斯坦（Winstein）说，我语言的界限意味着我世界的界限。我认为在追求雄心勃勃的项目时，情况非常契合这句话。

英文原文

It's the same thing with like development now, right? I can make multiple apps and I have to think about why I like something and why I don't. And most of my time in art school was just thinking, man, like I can make things and I can look at them and I just say, I don't like how this looks. And what you I was noticing was that my taste and my judgment exceeded my skill. I could not do the thing I liked, right? And I can't just look in the mirror and tell myself like make it pop a h 100 times and and know that it's going to be better. I have to go out and develop that language, right? People say like taste is the moat, but I think I I I generally hate that line because this doesn't tell you what taste really means, right? To me, taste is the ability to describe what you like and predict what others will like, right? and building this model of the world and of other people. That's the new thinking, right? Winstein's the limits of my language means the limits of my world. And I think that's very much the case on pursuing ambitious projects.

你知道，关于运用语言有一个有趣的例子。我不知道史蒂夫（Steve）在不在房间里，但他几天前发了一条推文，只写着：“找出我应用中所有隐式的状态机。”我还得去谷歌搜索那是什么意思，对吧？突然之间，所有的评论都在说大家的应用运行速度都提高了大约 20%。这些现在都是非常简单的方式，纯粹用来建立一些语言，去理解好东西长什么样，以及如何把东西做得更好。

英文原文

You know, it was a funny example of just like the use of language. I don't know if Steve is in the room, but he made a tweet a couple days ago that just says, "Find all the implicit state machines in my app." I had to go Google what that meant, right? And all of a sudden, all the comments are just how like everyone's apps are like 20% faster. These are now like really simple ways of just developing some language to understand what good looks like and how to make things better.

如果你想想机器学习中的模型进展，有一个非常有用的概念叫作生成器-判别器差距（generator discriminator gap），对吧？你有两个模型。一个给模型打分，另一个实际生成输出。发生的情况是，如果评分者太强，模型就学不会；如果生成器太强，判别器就无法帮助模型提升。

英文原文

If you think of like model progress in machine learning, there's a really helpful concept called the generator discriminator gap, right? You'll have two models. One grades the model and then one actually creates the outputs. And what happens is is if the grader is too good, then the model can't learn and if the generator is too good, the discriminator can't help the model improve.

我们现在就是那个判别器，对吧？随着模型能力提升，在某个时间点，也许模型的输出会达到你自己的品味水平。到那时就很难再真正取得提升了。然后为了做出更好的东西，你必须走出去建立那种语言，懂得什么是好的。这里的失败模式就是无法辨别模型是否能够提升，对吧？再次强调，这意味着我们必须走出去重新学习，我们必须重新思考。这正是我在过去一年里仅仅为了构建软件而花时间学习的一大堆东西的例子，对吧？比如当我构建这些应用时，我意识到，天哪，这些网站看起来真丑。比如我需要学习排版。比如为什么所有的东西都是斜体？我完全不明白。于是我不得不去学，我花了很多时间学习排版。然后就是关于信息如何呈现以避免看起来显得粗糙。再后来就是关于动效设计，以及我怎样才能让事物以优美的方式动起来。我得以花很多时间学习动画，然后学习交互模式以及是什么让一款应用感觉优美。这真的很有趣，对吧？这又是花更多时间去学习，以此来做出出色的东西。

英文原文

We are now that discriminator, right? As model capabilities go up, at some point maybe maybe the model's outputs hits your own taste levels. Then it's really hard to actually improve. And then to make things that are better, you have to go out develop that language and know what good looks like. And the failure mode is here just not being able to discern whether or not a model can improve, right? And again, what this means, we have to go out and learn again. We have to think again. This is just an example of a bunch of things I spent my past year learning like in just building software, right? Like as I build these applications, I realized, oh man, these websites look really ugly. Like I need to learn about typography. Like why is everything itallic? Like I don't understand. And I had to learn I spent a lot of time learning about typography. Then it was about like how the information was presented to avoid looking like slob. And then it was about like motion design and how can I make things move in a beautiful way. And I got to spend so much time learning about animation and then learning about like interaction patterns and what makes an app feel beautiful. And this is really fun, right? And this is again spending more time learning in order to make great things.

最重要的是，有能力描述事物才是让你脱颖而出的关键，对吧？如果你想想那些紫色界面的应用，它们之所以存在，是因为制作这些应用的人缺乏眼力。他们根本无法描述哪里出了问题，他们觉得这样就可以了，对吧？而这种语言上的局限正是我们所面临的问题。

英文原文

And most importantly, having the ability to describe things is what allows you to differentiate, right? If you think about these purple apps, the reason these existed was because the people making these these applications like didn't have the site. They couldn't actually describe what was wrong and they thought this was okay, right? And this limiting in language is sort of the problem that we face.

那么现在的问题就是，好吧，我们该如何建立这种新语言？老实说，我现在几乎被灌输的道理就是：如果你想要好的品味，你就必须去吃。你

英文原文

And then so the question now is okay, how do we develop this new language? And honestly, the line that I've sort of been like punished to now is just like if you want good taste, you have to eat. You

走出屏幕体验世界，提升审美品味

必须走出去探索事物，对吧？大家现在都在凭借感觉（vibe coding）做电子游戏。也许答案并不只是去让模型做得更好，而是去玩一玩电子游戏，理解什么感觉才对，什么样的动画让人感觉舒服，什么样的故事有分量，自己到底想讲什么样的故事，对吧？如果你在做工具，就去试试别人的工具和应用，明白自己喜欢什么。去看美好的事物。我一直在用 AI 做很多 3D 打印，但所有东西看起来都像是粘在一起的长方形。我就一直说“让它更美观一些”，对吧？那真是一种让人非常沮丧的经历。于是我开始去观察跑车，去观察工业设计。我开始去商店看各种物件，告诉自己去描述并理解什么是好看的弧线。这又是一个非常有趣的体验。现在我的很多设计都变得无比精美——至少达到了我认为的美。对吧？

英文原文

have to go out and explore things, right? People are like vibe coding video games. Maybe the answer isn't just to like tell the model to do better, but like play some video games and understand what feels good, what what animations feel good, what stories matter, what stories do I want to tell, right? If you're building tools, try other people's tools and applications and understand what you like. Look at beautiful things. I've been doing a lot of like 3D printing with AI and everything just looks like rectangles glued together. And I just kept saying like make it more beautiful, right? And that's a really frustrating experience. And I started looking at like sports cars and like industrial design. I started going to stores and looking at things and go like let me describe and understand what like good curves look like. And again, it's like a really fun experience. And now more of my designs are like incredibly beautiful for what I think is beautiful. Right?

我最近一直在协助做营销，然后我意识到，虽然我一直在告诉营销团队应该多用用 Codex 应用，但我自己却没有多看广告；而当我看了更多电影时，我意识到有很棒的故事可以去讲，一切就再次变得更好了，对吧？这正是使用这些工具令人非常兴奋的部分，对吧？去思考为什么我喜欢这些事物、为什么它们很奇妙、为什么它们让人感觉良好，现在成了我消磨时间的方式，对吧？所以我并没有减少思考，而是在进行更多思考，这真的丰富了我的世界，让我以不同的方式看待事物。

英文原文

I've been helping with like marketing and I realize that like I tell the marketing team that they should use like the Codex app more but I'm not watching more ads and as I'm watching more movies I realize there's great stories to tell and everything becomes better again right and this is kind of the really exciting part of using these tools right and like thinking about why I like things and why things are amazing and why things feel good now is how I spend my time right so I'm not thinking less I'm thinking more and it's really enriching my world and making me see things differently

如果你在带领团队，我认为这里最值得汲取的一点是：如果你真的相信 AI 让你更有生产力，我认为你应该对自己和团队的其他成员负责，腾出空间走出去探索事物，尝试新的应用，真正地走出去体验世界，对吧？如果你是一个能做出优秀作品的人，也可以去激励他人，对吧？这样你才能带着自己的观点回来。所以走出去，去获得灵感。学会成为一个更好的鉴赏家，并认识到你实际上并没有减少思考，而是在思考得更多。

英文原文

And if you're running a team, I think that the one takeaway here is that like if you really believe that AI is making you more productive, I think you owe it to yourself and the rest of the team to make space to go out and explore things and try new applications and and just actually go out and like experience the world, right? And if you are someone that makes really great work, inspiring others as well, right? And that's how you can come back with a point of view. So go out and be inspired. Learn to be a better critic and recognize that you're not actually thinking less. you're thinking more

好奇心与跨学科融合

最后一点其实就是保持好奇心——真的非常、非常好奇地去追求新想法。比如当你们思考自己作为软件工程师的角色时，我希望大家思考的一个问题是：当年为了从事软件行业，你放弃或搁置了哪些东西？你在作为软件工程师的过程中培养出的判断力和经验，还能用在什么地方？因为我觉得大家都在担心被取代，但实际上我认为很多地方都需要人。

英文原文

and the last thing is actually just being curious really really curious to pursue new ideas like one question I want you guys to think about as you are thinking about your role as software engineers is like what were the things you gave up or put aside to pursue software right where else could use your judgment and your experience you've developed as a software engineer right because I think people worry about being replaced but really I think people are needed in way more places

想想看，当铅笔被发明出来，或者雕塑变得更容易的时候，创意领域的各个不同行业都变得非常跨学科，对吧？完全合情合理的是，如果你是一名家具设计师，你也可以成为一名建筑师，你还可以拍电影、设计儿童玩具和游乐场，对吧？比如米开朗基罗，他既是雕塑家、画家，又是建筑师和诗人。在当时，你能做所有这些事情是完全讲得通的，对吧？这是一个创意的世界。你发明了铅笔，就能做很多非常惊艳的事情。但随着技术的发展，想要兼顾多行变得更加复杂了。我不可能去读四个博士学位；就算能读完，可能我都50岁了，对吧？因此，掌握多项技术技能已经变得极其困难。

英文原文

if you think about when the uh you know invention of like the pencil or or sculpting became more easy. All these different industries in the creative world became really very much multi disciplinary, right? It completely made sense that if you were a furniture designer, you could be an architect and you could like make a movie and design kids toys and playgrounds, right? Like Michelangelo was a sculptor and a painter and an architect and a poet. You know, it completely made sense that you could do all these things, right? And this is a creative world. like you invent the pencil, you can do a lot of really amazing things. But with technology, it's become even more complicated to do so. I just can't get like four PhDs. And if I could, like maybe I'll just be like 50, right? And so being technical has become incredibly difficult.

但当你真正将这些不同的行业进行跨界融合时，奇妙的事情就会发生，对吧？分子生物学的创始人曾经是一位物理学家，对吧？一位数学家开始领导人类基因组计划，对吧？对吧？而与磁体相关的数学知识，正是用来帮助理解神经网络的工具。但这在现在变得超级困难，因为你需要掌握太多的信息。对吧？即使在我们的世界里，我们也有很多 AI 在帮助我们设计芯片、编写软件。这些同样是非常跨学科的追求。

英文原文

But when you do cross-pollinate these different industries, amazing things happen, right? The physicist was sorry, the founder of molecular biology was a physicist, right? A mathematician started leading the human genome project, right? Right? And the mathematics around magnets was what you is what is used to help understand neural networks. But this is now like super hard to do because you need so much information. Right? Even in our own world like we have a bunch of AI helping us design like chips and writing software. And these again are just very multi-disiplinary pursuits.

如果你看看在外部真实世界中拥有抱负意味着什么：有一位机器学习顾问，正在利用 AI 和分子生物学来帮助治疗自己的狗所患的癌症。北海道有一位公务员，使用聊天和计算机视觉模型制造了机器人来实现农场自动化，对吧？这种跨界融合，这种创造这些事物、学习并在不同路径之间相互借鉴的意愿，才是真正令人振奋的，对吧？今天早些时候我好像看到有人用 AI 制造了更多的机器人，还用来发现系外行星，对吧？对吧？正是这种好奇心，以及 AI 所赋予你的赋能和门槛的降低，让你有能力在这些领域树立远大抱负，对吧？

英文原文

If you look at what it means to be ambitious outside in the real world, there's a machine learning consultant using AI and molecular biology to like help treat their dogs cancer. There was this uh public servant in Hokkaido who used like chat and like computer vision models to build like robots to automate their farm, right? This cross-pollination and this will to create these things and learn and cross-pollinate across these different paths is what's really inspiring, right? I I think earlier today I saw folks are using AI to like make more robots and also like discover exoplanets, right? Right? And it's that curiosity and that enablement of what AI allows you to do and lowering that barrier is uh what allows you to be ambitious in those fields, right?

所以当谈到思考自己是否会被取代时，很大程度上是因为你现在拥有了每个领域中每个人都需要的技能。这不仅仅是编写更多应用的问题，而是真正去思考：当年为了学习软件工程，我在大学里没有去追求什么？我该如何重返那些领域，而 AI 又能怎样让这些事情变得更容易？对吧？你同样可以建立新的语言体系，理解是什么让事物变得绝妙，然后去追求它们。这种做法带来的巨大乐趣，以及这些跨界艺术家之所以如此成功的原因，就在于当他们探索新媒介时，重新像个孩子一样的状态意味着你有提出看似不可能之事的纯真。对吧？这种跨界之所以有效，是因为如果你拥有在软件领域建立的判断力并涉足另一个领域，这意味着你可以提出别人可能认为不可能的问题，并且你有勇气借助 AI 帮助自己达成目标。对吧？因此，既能在一个领域重新找回新手的状态，同时又拥有 AI 领域的全部经验，这其中有着巨大的益处。对吧？

英文原文

And so when it comes to thinking about whether or not you can you're going to be replaced, a lot of it really is the fact that you now have the skills that everyone needs in every field. It's not just about like writing more apps, but actually thinking about, you know, what were the things I didn't pursue in college to pursue software engineering? What how can I go back to those worlds and how can AI make those things easier? Right? You can again develop new language and understand what makes things amazing and then go go pursue those things. And the great joy of this and the reason why these like multi-disiplinary artists are so successful is as they pursue new mediums, the act of being a kid again means that you have the naivity to ask for what might be impossible. Right? The reason this crosspollination works is if you have the judgment you've built in software and you pursue another field, it means that you can ask a question that people might not think is possible and you have the courage to use AI to help you get there. Right? And so there's great benefits in being able to feel new again in a field while all having all this experience in AI as well. Right?

用 AI 赋能个人兴趣与跨界实践

在我的个人生活中，我一直在学打鼓，一直在开发定制的击鼓软件。AI 在帮我学习如何使用 Ableton。我把我的架子鼓连接到了 MIDI，并在那里学习。我还一直在用 Computer Use 来教我以前从未真正有时间去学的新软件。我甚至一直在用 AI 帮助我进行实体工程设计，比如 3D 打印零件、制作芯片。同样，这些都是我过去一直感到好奇却从未有过时间去做的事情。而且，拥有 AI 的一部分不仅在于把时间还给了我，其他所有事情也都被加速了，对吧？因此，拥有抱负就是去探索这些追求，而不仅仅是消耗更多的 token 来构建一个应用程序。

英文原文

In my personal life, I've been like learning the drums. I've been building custom like drumming software. AI is helping me learn how to use like Ableton. I'm connecting my my drum set to MIDI and learning there. You know, I've been using computer use to teach me new software that I never really had time to learn. And I've even been using AI to help me design like physical soft uh physical engineering. So, I'm 3D printing parts, building chips. And again, these are things I was always curious about but never had the time. And again, part of having AI is not only getting my time back, everything else is accelerated, right? And so, being ambitious is about exploring those pursuits, not just spending more tokens like building an app.

所以这也是我希望大家去思考的，对吧？再次想想，你搁置了哪些领域？现在有了 AI 带来的更多时间，你该如何去追求它们？

英文原文

And so this is also what I want you guys to think about, right? Again, what fields did you put aside? How can you pursue them now that you have more time with AI?

因此，当我说提高你的抱负时，我并不只是在说“去成为一名生物学家”，对吧？我真正想说的是，为了变得更有抱负，你现在工作的一部分就是真正受到启发，去追求这些事物，去学习那门语言，同时也去激励下一代。

英文原文

And so what I'm when I'm talking about like increase your ambition, I'm not just saying like become a biologist, right? What I'm really saying is that like part of your job now to become more ambitious is to be really inspired, pursue these things, learn that language, and also for that next generation to be inspiring.

对吧？过去每当有人问我如何学习编程时，我从来没有说过“你知道吗，你应该去读一本关于 Python 的教科书”。我总是说“去动手做点东西”，对吧？总是说去写一个天气应用，去写一个笔记应用，找到一个真实的问题并解决它。而现在，这些问题的范围要大得多了。所以作为软件开发者，我认为我们现在的职责不仅是做出令人赞叹的东西并分享它们，而且还要激励下一代抱有更多的希望，对吧？因为也许软件正在发生变化，但实际上我们能做出贡献的事情将会多得多，对吧？我们到处都能看到这一点。比如 Codex 应用和站点产品，这些全都是充满热忱的心血之作。Codex 应用、站点产品全都是由真正关心这些事物的人打造出来的。他们在网上与人交流，不断解决这些问题，对吧？现在我们的产品经理可以做工程，我们的工程师可以做设计。这种跨界融合正在激励其他人也做同样的事情，并承担起

英文原文

Right? When people used to ask me how to like learn how to code, it was never a case where I said, you know what, like you should go read a textbook on Python. It was always build something, right? It was always go build a weather app, go build a like note-taking app, have a real problem and solve it. And now the scope of these problems are much bigger. And so as software developers, I think our job now is to not only build amazing things and share them, but also inspire the next generation to have more hope, too, right? Because maybe software is changing, but really we're going to be be able to contribute to so much more things, right? And like we see this everywhere. Like for example, the Codex app and sites, these are all things that were acts of love. Codex app, the sites product were all built by people who really cared about these things. They're talking to people online and working through these things, right? And now our PMs can do engineering, our engineer can do design. And this cross-pollination is inspiring others to do the same and take ownership of

行得通，对吧？这里存在的大多数东西之所以在这里，是因为人们坚韧不拔，不断尝试，并希望事物变得美好，你懂的。归根结底，我们 OpenAI 的使命是造福所有人，而这最终的含义并不只是让 OpenAI 拥有并控制所有这些 AI，去使用智能体集群之类的，还包括确保每个人都能拥有这项技术，从而能够以自己的方式做出贡献。

英文原文

that work, right? And most of the things here that exist are here because people are tenacious and are trying and want things to be beautiful, you know, and ultimately our mission open AI is to benefit all and ultimately what that means isn't just having open AI have all these AI to to control and you know use agent swarms or whatever, but also making sure that everyone has this technology so they can contribute in their own ways.

所以基本上就是这样。想想你所拥有的东西。走出去，获取灵感，形成去描述美好事物的样貌所必需的语言，并且确保你真的保持足够的好奇心，花时间去追求这些新的兴趣。然后你就会变得更有远大抱负。所以，我非常期待看到大家正在构建的东西，欢迎到展位来聊聊。谢谢大家。[掌声]

英文原文

And so that's basically it. you know, think about the things that you own. Go out, get inspired, develop the language you need to describe what beautiful things look like, and just make sure you're actually cur curious enough and take the time to pursue these new interests. And then that's how you become more ambitious. So, I'm really excited to see what you guys are building and uh come by to the booth to chat. Thank you. [applause]

从创建 Flask 和 Jinja，到探索 AI 驱动软件开发的未来，我们的下一位演讲者帮助塑造了开发者构建 Web 的方式。请欢迎 Arendelle 的首席技术官 Armin Ronacher。

英文原文

From creating Flask and Ginga to exploring the future of AI powered software development, our next speaker has helped shape how developers build for the web. Please welcome Armen Ronaher, CTO of Arendelle.

工坊与软件工厂

大家好。我的名字是 Armin Ronacher，我想和大家分享一些我们在使用智能体构建软件过程中学到的经验教训。特别是，我想提出一个强有力的论点，那就是比起建造工厂，不如更多去建造工作室或工坊。

英文原文

Hello. Uh, my name is Amin Rona and I want to share with you a little bit the lessons that we have learned from building software with agents. And in particular, I want to make a strong case for um not so much building the factory as building the artiller or workshop.

我想通过我们自身使用的一些数据来展示这一点，但我特别想解释一下，在建造软件工厂之前，建造工作室或工坊究竟是什么意思。

英文原文

And um I want to show some data with this from our own usage, but in particular I want to explain what I mean by building the artiller or the workshop prior to building the software factory.

我想这里的每个人都已经使用智能体相当长一段时间了，大家可能已经注意到，就我们能用它们做些什么而言，我们正处于某种指数曲线上。但我在这里想作出的比较是，你可以成为一名出色的艺术家，使用从数控机床（CNC）、3D 打印机到极佳的电动工具等最新最棒的技术，这些技术可以提高你用艺术进行创作的抱负。我认为在某种程度上，我们也可以这样看待软件。

英文原文

And um I think everybody here is being use has been using agents for quite a while and you have probably noticed that uh it is we're sort of on an exponential curve in in what we can do with them. Um but the comparison that I want to give here is that you can be an amazing artist and use the greatest and latest technology from CNC machines to 3D printers to just very good power tools and it can increase the ambition of what it can do with your art and I think we can also look at software a little bit in that sense.

所以重点不在于构建多得多的软件，而在于构建更宏大、质量更高的软件。接下来我会给大家展示一些我的具体意思。

英文原文

Uh so less about building much more software and just building way more ambitious and high quality software. Um so I'm going to show you a little bit of what I mean by that. Um

大家可能知道 Arendelle 正在开发 pi 编码智能体。因此，在我们看待这件事的方式上，显然带有一点我们自己的利益考量。

英文原文

and um you might know that Arendel is working on the PI coding agent. So we have obviously a little bit of a horse in the race here of how we're looking at this. Um

但我认为相当有意思的是，pi 编码智能体的核心框架这一年来基本没有改变。它今天拥有的工具，与去年 12 月时完全一样。我认为这很有意思，因为它能力的增强完全仅仅归功于模型变得越来越好。

英文原文

but I think what is probably quite interesting is that the pi coding agent has not really changed in a year on the core harness. It still has the same tools today as it did uh in December last year. And I think this is interesting because it has become much more powerful purely as a fact of the models getting better. Um

所以，在框架的核心功能没有任何落地的更改的情况下，我今天能用 pi 做到的事情能力大大增强了。我认为这是一个值得反思的有趣现象，因为这意味着即使我们在实现方式上没有做出任何重大变更，我们所有人也同时都在变得更强大。

英文原文

so I'm much more capable today of what I can do with pi without any changes landing in the core functionality of what the harness is doing. And I think this is an interesting thing to reflect on because it means that we are all simultaneously getting better even without landing any significant changes in how we're doing this.

大家可能还记得，在过去大约一年半的时间里，出现过很多关于如何更好地给机器编写提示词，或者如何在测试框架上搞一些巧妙花样的想法。对我来说，其中极少数在今天仍然是必需的。这里有一个很好的例子：我曾经非常喜欢 PI Auto Research，它是一个插件，基本上能让智能体朝着像提升性能这样的目标去推进。但如果你使用当今最先进的模型，你其实会发现整个插件已经不再需要了。所以模型内部正在发生大量的变化，极大地改善着我们的工作方式。

英文原文

And um you might have remembered throughout the last year and a half maybe that there were a lot of ideas of how you can prompt the machines better or how you can do some interesting tricks with the harness. Very few of them are still necessary for me. A good example here is that I was in love with PI Auto Research which was a plug-in that basically allowed an agent to drive towards a goal like improving performance. But if you use a present day state-of-the-art model, you actually discover that entire plugin is no longer necessary. So there's a lot that is just happening in the models that is dramatically improving how we're doing. So these machines are getting better.

使用数据与瓶颈转移

我只想通过几个图表来展示这一点，这些图表基本上来自我的 pi 会话，我过滤了它们，只保留默认设置以及修正的总次数，我用它进行了分析。随着每一个越来越好的模型推出，我不得不进行的修正总次数一降再降。所以与去年 12 月相比，使用最新模型时我对智能体的反驳和干预少了很多。

英文原文

Um and I just want to show this from a couple of graphs from basically my PI sessions where I filtered them out to be have nothing but the default set and the total amount of corrections and I used Jeff to analyze this. The total amount of corrections that I had to do uh got down and down and down with every single model that got better and better. So I'm I'm pushing back a lot less against the the the the agent um with the latest model than I did in December last year. Um

我中止它的频率也是同样的情况。而在我分析会话的方式中，我认为最有趣的指标是平均时长。这是第 90 百分位数（P90）：去年 12 月使用 Opus 4.6 时，智能体响应我给它的任务运行的第 90 百分位数时长大约是 2 分半钟；而现在使用最新模型，运行的第 90 百分位数时长达到了 14 分钟以上。这确实在某种程度上改变了你想如何使用测试框架，但在 pi 里面真的同样什么都没变，而我仍然看到了我自己使用方式上的这种行为差异。

英文原文

and the same thing with like how often I'm aborting it. Um and I think this to me was the most interesting way in which I sort of analyzed my sessions which is the average. So this is the P90th percentile but on the 90th percentile of how long an agent was running in response to a task that I gave it with Opus 4.6 in December last year was about 2 and a half minutes and now with the latest models they are on the 90th percentile running for 14 minutes and more and that does actually change something about how you want to work with a harness but really again nothing in pi has changed and still I'm I'm seeing this sort of different behavior of of how I'm using this um

如果你还在使用智能体并观察它在做什么，你可能也以某种形式注意到了：当今的智能体使用 bash 的可能性远远大于提供的任何其他工具。但是再次强调，我们在框架的配置上没有任何改变。这纯粹是模型本身涌现出多得多的行为。这里缺少了 Astra 和 Soul。从我的测试来看，它们其实看起来和 Opus 非常相似。只是我的测试集中没有足够的数据来很好地展示它们。

英文原文

and this probably you have noticed in one form or another if you're still using an agent kind of see what it's doing you probably noticed that the present day agents are much more likely to use bash than any of the other tools that are being provided. But again, nothing has changed in how we configure the harness. It's just that there's a lot more emergent behavior coming from the models. What's missing here is Astra and Soul. They actually look very similar to Opus from my testing. I just didn't have enough data in my test set um to show this nicely. Um

大家会注意到这里有一点紫色的小东西。我想在最后谈谈这个，也就是某种“代码模式”（code mode）正在慢慢爬升。

英文原文

I you will notice that there's a little purple thing here. I want to talk about this in the end, which is sort of code mode is sort of slowly creeping up. Um

但这一切的核心在于，哪怕你现在拥有一套现成且表现不错的基础设施，它仍然在随着模型的发展而改变。过去一年里没有改变的——至少对我来说是这样，而且我认为对很多人来说也是这样——那就是我们仍然是人类。我们仍然受限于我们自身的基本能力。

英文原文

but the the core of this is um even if you have a piece of infrastructure nowadays that is sort of there and is doing a good job, it's still changing as a result of the models. Um what has not changed in the last year um at least this is the case for me and I think this is the case for for a lot of people is that we're still humans. We're still limited by our fundamental capabilities. Um,

随着我们抱负更加宏大、用这些机器做的事情越来越多，我不想说我们是瓶颈，但我们确实是给它们能做的事情定下节奏的因素。尤其是，至少对我而言，人类的注意力是限制性因素。我在编码智能体如何在团队中部署的过程中很大程度上注意到了这一点：我们所拥有的、可用于工作的注意力基本上是一个恒定数量。

英文原文

and as we are being more ambitious and we're doing more with these machines, um, we're I'm not I don't want to say like we're the bottleneck, but we are the thing that paces what they can do. Um, and in particular, human attention is for me at least the limiting factor. And I'm noticing this with largely in how coding agents deployed in teams. um that this is sort of a stable amount of like attention that we have that we have to work with

使用这些智能体启动一个新项目变得异常容易。我认为特别是在新模型推出时，大家都会极度兴奋地去尝试能做的最新、最棒的事情，从 3D 游戏到自动化各种事务的新方式；但维护它们并没有真正变得更容易。

英文原文

and um starting new project is getting incredibly easy with these agents and I think like particularly when a new model comes out we're all super excited of trying the latest and greatest thing you can do from 3D games to um new ways in which we automate things but maintaining them has not really gotten easier um

而且我们还在越来越重地给很多外部系统带来负担，这些系统在很多方面其实本不应该卷入这场变革之中。我想大家都知道推送到 GitHub 上的提交多得多，代码仓库也多得多；但我知道有不少人在互联网上拥有根基深厚的老网站，这些网站为智能体赋予了新生，而它们完全没有为直接来自我们的大量新访问做好准备。所以我在这里只想提醒一点：当我们推动事物向前时，我们所有人也同时意外地在推动一些智能体正在做、但外部系统还没真正准备好的事情。

英文原文

but we are also increasingly taxing a lot of external systems that actually in many ways are not supposed to be part of this revolution. Um, I think we all know that GitHub has a lot more commits being pushed up there and a lot more repositories, but I know quite a few people who have wellestablished old websites on the internet that found a new life for agents and they are totally not ready for the new volume that's coming from from us kind of directly hitting them. So I I just want to caution a little bit here in saying that when we are pushing things, we're also accidentally simultaneously all of us pushing things um which we're not really ready for for what agents are doing. Um

对我而言，我认为这是一个有趣的图表：一年中我主动发送的提示词数量基本上是持平的。我知道某些月份会稍高一点，某些月份会稍低一点，但基本上都在每天发送 120 到 190 个左右的提示词之间轮换。

英文原文

and this to me I think is an interesting graph because the number of pro prompts that I'm actively pushing over the year is is sort of flat. I know that like in some months it's a little bit higher, in some months it's a little bit lower, but they're they're alternating between 120 190 something like this prompts uh that I'm pushing per day. Um

但大家可以看到，智能体为我活跃使用的时间，从少于我的工作日时长，变成了我工作日时长的数倍。而且这个 11 月甚至还没结束，对吧？所以大家可以看到这正在呈指数级曲线上升。

英文原文

but you can see that the time that the agent was active for my use went from like less than my working day to many times my working day. Um and this November is not even over, right? So you can see that this is taking an exponential curve up. Um,

而这里是项目的数量。这是我过去一年创建的 Git 仓库数量。显然这在累积上升。我有 141

英文原文

and this here is the number of projects. This is Git repositories that I've created over the last year. And this is like obviously cumulatively going up. So I have 141

我启动的那些不同项目。但是我定期持续提交的项目数量——也就是说我提交超过五天且还会再回过头来提交的项目——基本保持平稳，大概在 10 到 15 个左右。

英文原文

different projects that I started. But the number of projects I'm committing on a regular cadence to um which means like I'm committing for more than five days to and I'm returning to them also um is pretty much flat with with between 10 15 something like this.

我也在一群朋友中做过这种非正式的调研，这似乎是一个普遍趋势：实际定期维护某个项目其实很棘手。我和一位朋友讨论过这事，跟这个图表有点关系，他说：“其实我有一个开源项目，现在完全自动化了。有 issue 进来，设置了一个 agent 自动 review、提交并反馈回去。这也许是一种扩展规模的方式，把人从整个流程中剥离出去。”但实际上，他后来也把这个系统关了，因为最终他没那么在意那个项目了，而且他想把自己的 token 配额留给其他事情用。所以，我们对开源项目的关注度非常重要，即便我们可能已经把它完全自动化了。如果我们不再使用某样东西，我们往往就不会再去积极维护它了。

英文原文

Um and I have I have done this sort of an informal survey with a bunch of my friends as well and this seems to be a common trend that actually maintaining something regularly is tricky and um I had this discussion with a friend of mine uh and sort of in relation to this graph and he said like well actually I have an open source project that is now fully automated. So, uh, there's a issue tracker coming in. There's a there's an agent that's set up to auto review, commit, and send it back. And that is maybe a way to scale up. So, you take the human out of the whole thing. But actually, he also turned it off because eventually he didn't care about the project as much anymore. And he wanted his token specs to use them for other things. Um, so our attention to an open source project matters even if maybe we have automated entirely. So if we stop using something where we have a tendency to no longer actually actively take care of it.

真实技能与 Agent 技能

所以，如果我们正在努力推动——而且我认为首先要弄清楚如何搭建一个真正优秀的工作坊来构建高质量的软件——这里就存在一个“什么才是最重要的”问题。我认为对我来说最重要的三件事，一方面是切实知道什么时候该对 agent 的做法提出反对，告诉它这是错误的做法。我认为这是一项至关重要的技能，因为 agent 获得的奖励仅仅来自于完成一个目标，而它是否完成了这个目标是非常容易观察的。我们可以把这看作类似于在训练过程中 RL 环境对模型的奖励。但这并不意味着实际交付出来的成果，就是我们在较长时间跨度内打心底里满意的。随着我们提交的 commit 越来越多、软件规模越来越大，这就变得很棘手。软件很容易就会变成各种特性的“辛普森汽车”式堆砌。但在系统的内部复杂度不发生爆炸的情况下，真正让各个部分内在地契合在一起，依然需要我们在它工作时与其协作配合。

英文原文

Um, so if we're pushing towards and I would make this argument first figuring out how to make a really good workshop to build really high quality software, there's a question of what matters. And I think for me the three most important things to matter is on the one hand to to really know when to push back against what an agent is doing, telling it that this is the wrong way of doing it. And I think that this is a crucial skill because an agent is really rewarded in just completing a goal. And it is very easy to look at if it has succeeded in this goal or not. Um and this we can sort of see it in the same way as the the RL environment maybe is rewarding the the model during training. Um but that doesn't mean that what is being actually delivered is something that we are intrinsically happy with over a longer period of time. And this becomes tricky the more and more commits we're doing, the larger and larger the software is going. It's very easy to basically create the Simpson car of all the features in a software. Um but actually having something that intrinsically fits together where the internal complexity of a system does not is like um explode uh still requires us to um to to work with it as it's working.

我喜欢把这称为“真实技能与 agent 技能”。有经验的软件工程师与 agent 配合工作，跟没有经验的软件工程师相比，显然存在差异。对我来说最真实的体现是，我儿子现在会在我们一起做的一个项目上给我提 PR。他的贡献在质量上有着非常明显的差异，因为他不是软件工程师。但我们提示 agent 时用的是同一套系统，对吧？我自己遇到过这种情况，尤其是从许多其他人那里也经常听到这类情况：如果你身处一个真正关键的场合，房间里的其他人基本上只是去找 agent 来回答问题，而不再真正理解系统在做什么，或者不再理解他们正在解决的核心问题，这其实是一件非常令人沮丧的事。所以，依然理解自己所做的事情具有很大价值。这也意味着你得在某种程度上克制自己的操作节奏，从而始终在你的理解范围内开展工作。

英文原文

And I I like to call this like true skill versus agent skill. Um there is clearly a difference between someone who is an experienced software engineer working with an agent and someone who is not an experienced software engineer. And um I think for me the the truest form of this is that uh my son is now sending me pull requests on a project that we're doing together. And there's a very clear quality difference in what his contributions are because he is not a software engineer. Um but we're using the same system to prompt it, right? Um and I have I've noticed this myself in situations, but I' I've heard this also particularly from a lot of others. It is actually a very frustrating experience if you're in a situation that really matters and other people in the room are basically just going to the agent to answer a question and no longer have a really under good understanding of either what the system is doing or that the fundamental problem that they're working on they actually comprehend. Um so there is a lot of value in still understanding what you're doing. Um that also means that you kind of have to moderate the speed in which you're doing things to still work within your understanding.

随着我的 commit 数量基本上开始飙升，我开始采取的应对方式之一就是：审视我的 session 轨迹是如何发展的，我能否在结束前的某个环节介入，找到一种让我能持续评估当前进展的方法。举个例子，我开始提示 agent 在进入验证环节（通常是在进度的 70% 左右）之前，开始生成排版精美、简明扼要的高管摘要 HTML 页面，这样在它完成的过程中我就能真正理解它在做什么。特别是使用较新的模型，生成的文本读起来已经没有那么糟糕了。但这是我找到的一种平衡手段，因为在很多情况下我基本上不再去读代码了。

英文原文

And one of the ways in which I started doing this as my commits started going through the roof basically was I I looked into how are my trajectories of sessions going and can I hook myself into a part before the end of it to find a consistent way for me to evaluate what's going on. And so for instance, I started um basically prompting the agent that before it's going towards the verification, which is typ typically around 70% of it to start producing a really nice short executive summaries in a nice type set HTML page, they can really understand what it's doing as it's finishing. Um particular with newer models, it's no longer quite as horrible to read the text that comes out of it. Um but this is one way in which I found to balance out that I'm basically no longer reading the code in a lot of cases.

标准与协作的困境

我认为对我来说越来越重要的一点，也是要带一点指引：不要反复重写所有软件。我非常推崇少用依赖、自己多写代码。出于这一点，我非常喜欢 agent 带来的变革。但是在德语里我们有个词叫“Wildwuchs”，意思基本就是野蛮生长，而在本应协同工作的软件中，野蛮生长可能会带来很多极其负面的后果。你可能见过最近流传的一篇博文，讲的是一个精心构造的 tar 归档文件，不同的 tar 实现会以不同的方式去读取它。我认为现在这成了一个问题。为了能够协同工作，我们非常需要标准，需要通用的规范，需要通用的文件格式；而我们现在基本上能让 agent 去实现我们想要的任何格式。但这些差异是必然会出现的。而真正阻碍我们达成统一标准、阻碍我们共同协作完成某件事的，往往不完全是技术挑战，更多的问题在于：人们如何就该文件格式中应包含什么、应如何实现以及应如何解释达成一致，因为对它可能会有多种解读。

英文原文

Um and this I think is something that increasingly is more important to me is to to to also show up with a little bit of a a guidance towards um not writing all the software over and over. Um I'm a huge believer in using fewer dependencies and writing more code yourself. So I really like the agent revolution for for that element of it. Um but in German we have this what called vuk which basically means uncontrolled growth and uncontrolled growth in software that is supposed to work together can have a lot of really negative consequences. Um, you might have seen a blog post that was going around recently about a tar archive that is has been crafted in a way that different tar implementations read it in different ways. And um I think this is a bit of a problem now. Um we for us to work together we we really want to have standards. We want to have uh common specifications. We want to have common file formats and we now can basically make an age and implement any format we want. But there are going to be these differences. And what really makes it hard for us to agree on standards and what what makes us work together on something often is not a technical challenge as much as it is a problem of how do people agree on what is supposed to be in this file format and how is it supposed to be implemented and how is it supposed to be interpreted because there can be multiple interpretations of it.

我确实认为这些开放协议和文件格式是非常重要的方面。你可以设想未来我们人人都拥有自己的小型 Photoshop 仿制品，但我们再也无法互相发送文件格式了，因为彼此之间的偏离太远了。归根结底，这关乎人与人之间的协作依然像以往一样艰难。这在某种程度上拖慢了我们的步伐，对吧？我认为有时必须抵制那种直接让 agent 解决问题的诱惑，退后一步去弄清楚这是否属于我们应该与他人共同协作的那类问题。对我这个有开源背景的人来说，看到如今开源项目中实际的人类协作变得如此稀少，以及那么多项目最终产生无数个 fork，实在令人感慨——因为直接搞个 fork 总是比尝试将改动 upstream 上游更容易。

英文原文

Um and I really think that these open protocols and file formats are really important aspect. Um, you can imagine there is a future where we all have our own little Photoshop clones, but we'll no longer be able to send file formats to each other because we have diverged too much. Um, and this really comes down to this part that collaboration between humans is as hard as it has ever been. And this is slowing us in a way down, right? Um, and I think it's it's important to sometimes resist this temptation of just having the agent solve the problem and just taking a step back and figuring out if this is maybe one of those problems where we should be working together with other people. Um, it is for me coming from an open source background fascinating to see how little actual human collaboration is happening in open source projects now and how many more projects end up with gazillions of forks because it's just easier to have a fork happen than to try to upstream changes.

构建追求卓越与高质的软件

因此，我想极力主张去构建追求卓越的软件。我知道这很难。我认为最近我们看到了形式化验证软件的大幅涌现。同样非常明确的是，这本身也很难。技术栈还没有完全成熟到让这一套顺畅运作的地步。但我认为这是一个非常值得努力推进的领域，看看这是否有可能随着时间推移帮助我们构建更稳定的软件。看到那么多人想要构建在根本上运行更快、内存效率更高的软件，我真的非常高兴。我认为这是一个很棒的发展趋势。多年来我一直很苦恼的一点是，我的内存实际用在我认为有价值的事情上的比例竟然这么低。而现在出现了一个明显的趋势，人们尝试用原生语言重写东西，而不是像以前那样仅仅再套一层 Electron 外壳。

英文原文

Um, so I want to make a very strong argument for uh building software that strives towards excellence. And I understand that this is hard. Um we have I think seen a pretty big explosion recently of verified software. Um it's also very clear that this is in itself hard. The the the technology stack is not right there yet where this works. But I think it is one of those areas where it's really worth pushing towards and seeing if this is maybe something that can help us over time creating more more stable software. Um I really am very happy with seeing so many people wanting to build fundamentally faster and more memory efficient software. I think is a great development. Um, for years I was very frustrated with um, how little my RAM usage actually goes to things that I think are worthwhile. And now there's a significant trend of people trying to rebuild things in a native language rather than for instance just building another electron wrapper.

所以，所有这些都是非常酷的发展。但它们在很多方面仍然是在沿用我们过去创造的技术。我很喜欢 Rust。我喜欢现在这种构建内存安全软件的趋势。我想你们可能都接触过用优秀的模型去发现安全问题时会发生什么。所以对这方面有着强有力的理由。现在这也促使那些原本绝不会……

英文原文

Um, so all of those things are really cool developments. Um, but they are in many ways still building up on technology that we have created before times. Um, I really like Rust. I like that there is now a trend towards building memory safe software. I think you've probably all been exposed to what happens with um yeah good models finding security issues. So there's a strong argument towards that. Um and it it now has also led to people that would never

以前选择过那种语言的人可以实际去尝试一下，因为它带来的某些价值现在可以说是“免费”获得的了。

在最后一部分，我想稍微谈谈 harness（测试运行载体/运行控制框架）或许可以在哪些方面有所作为。这倒不是说我非要讲“看，这就是我们正在走的路径”，而是想让大家对我们推动这类任务变得更加长程运行时可能遇到的一些挑战有所了解。

英文原文

have picked that language before to actually give it a try because it delivers some value that now can in quote come for free. Um on the last part I want to talk a little bit about maybe there is something where harness can do something and this is less so much for me to say hey here's a thing that a path that we are on is to give you some idea of maybe where some of the challenges are as we're pushing this task to be much more long running.

首先，如果你的任务只需要两分钟就能跑完，你大概还挺乐意守在旁边照看着它；如果它报错了、过早结束了，或者没能完整运行到底，又或者你不得不重启 agent，那你就手动重启一下并让它继续，对吧？所以很长一段时间以来，我们基本上都是在这样一个大家觉得那样做完全可以接受的环境里工作的。但如果你现在拥有在夜间自动处理成千上万个 issue 的自动化 agent，你就会希望 agent 实际上能够自行重启。

因此我们非常推崇“持久化 agent”（durable agents）这一概念。同时也是指能够创建持久化工作流的持久化 agent，它们自我编排的方式使得你可以启动它、停止它、暂停它，并且能够把它恢复回来。这不仅是因为我们要面对更长程的任务，还因为我们当前所做的事情中存在大量的低效率。

我们大家在算力上都受到限制。在推理成本方面，我们都受到制约。如果能够在处理方式上更智能一些，我认为会非常有趣，我也希望有更多人向这个方向努力推进。例如，如果你按原始 token 成本付费，通过弹性调度（flex scheduling）你可以拿到相当可观的折扣。但这意味着可能好几分钟都没有任何 token 输出，对吧？那么这时候你就会想要挂起你的 agent，也想要挂起你的沙盒，从而切切实实地不浪费其他同样昂贵的资源。

所以在这一块上，我认为除了模型的质量之外——因为现在的模型已经变得越来越出色——我们或许也需要重新思考一下如何去更智能地使用这些资源。

英文原文

Um the first one is if if if you have a task that only takes two minutes to run, you're kind of willing to babysit it and if it breaks and uh it it ends early or it doesn't run to completion or you have to restart the agent to just restart it manually and tell it to continue, right? So it's like we have we have largely been working in environment for quite a while where that was acceptable. If you now have automated agents running on thousands of issues overnight, you want to have an agent actually be able to restart itself. And so we're pretty big believers on this concept of durable agents. Also durable agents that create durable workflows where they orchestrate themselves in a way where you can start it, stop it, you can suspend it, you can bring it back. And this not just because we have this longer running tasks, but also because there's a lot of inefficiency in what we're doing. And um we are all compute constrainted. Uh we are all uh constrained in terms of the cost of inference. And being able to be a little bit more intelligent about how we're doing this, I think is really interesting. And I wish more people would be pushing towards that. For instance, if you pay raw token cost, you get a pretty significant discount on flex scheduling. But that means that maybe for minutes no token is coming, right? And so then you want to suspend your agent, you want to suspend also your sandbox and actually not waste the other resources that are also expensive. So this is one part where I do think beyond the quality of the models because the models are getting so good now we might want to rethink a little bit of how we are using these uh resources intelligently.

第二点是，大家大概也知道，当 π 推出的时候，当时有很大的动力去认为 bash 就是你所必须运行的一切，而这在很大程度上依然是我们的信念。但现在我们也看到，开始涌现出编排子 agent 的需求，以及使用非 LLM 模型（比如决策模型，或者像 Jeff 这样的分类器模型）的需求。在这些场景下，你就必须开始组合工作流。

我认为这最初源于 Cloudflare 的团队，他们提出：与其把数以千计的工具硬塞进代码、塞进上下文里，那如果我们除了 bash 之外，再给 agent 提供 JavaScript 来运行，让 agent 原本原生执行的所有工具调用全部通过 JavaScript 来进行，这样它就能更好地对它们进行编排，情况会怎样？这种方式在许多方面已经从一种猎奇尝试，演变为了极其惊人且实用的东西。这意味着现在在很多情况下，harness——在 codex 里显然已经是这样了，在 π 以及目前许多其他的 harness 里你也可以做到——可以让 harness 去编排超大规模的工具调用，而无需经过效率低下的上下文。这确实需要对大家做事的方式进行一些重新思考。

而且这也允许你使用分类器模型。我认为这是另一个真正改变了我对某些技术能推向多远所持看法的关键部分，因为你不会因为没有激活的 KV 缓存而受到惩罚。而且它们也非常便宜、非常快。它们现在能让你把过去单纯因为成本太高或速度太慢而无法处理的某些问题，交由决策模型来搞定。

我认为大家对此应当采取的思考方式是：当你拥有代码模式（code mode）时，harness 所能做的是构建一个轻量级工作流，途中可以包含各种决策点。举例来说，如果你必须快速处理 3,000 个传入的客户支持请求，并且一路上必须做出决断——是将其分流给 agent 去跟进，还是需要客户提供更多反馈——你可以让 agent 基本生成一个按需定制的工作流，然后对于它所执行的每一个 if 条件判断，直接走 F 过去，这样你就能用你的 coding agent 在极短时间内、几乎毫不费力地处理掉成千上万个 issue，而在以往这本会花费你数个小时。

因此，伴随着这一点，我只想留下这样一个想法：如今既然我们什么都能造，也许我们并不应该盲目去造，而是我认为我们真的应该去推动打造比以往更加令人愉悦、而且在许多层面上更具雄心的软件。谢谢大家。

英文原文

And the second thing is um you probably know that when when when pi came out there was a pretty big um motivation to just say like bash is everything that you have to run and this is still largely what we believe. Um but now we also see that there is um an emergence of um desire to orchestrate sub agents, a desire to use models which are not LLMs like decision models or or classifier models like Jeff. And in those cases you need to start composing workflows. Um and um this I think originally came from the folks from Cloudflare say like in in instead of having thousands of tools being put into the code um into the context what if we just give the agent in addition to JavaScript in addition to bash JavaScript to run and have all the tool calls that the agent would natively do happen via JavaScript and then he can orchestrate them better. And this has gone from a curiosity I think in many ways to something that is fundamentally shockingly useful. And this means that now the harness in many cases and this is definitely the case already in codex uh you can do it in pi and many other harnesses at this point you can have the harness orchestrate very large invocations of tool calls uh without having to go through the inefficient context and this does require a little bit of a rethinking of how you're doing things um and it also allows you to to use classifier models and I think this is the other big part that really changed how I was looking at how far I can push certain technology because you're not being penalized for not having an active KV cache. Um, but they're also very cheap and very fast. And they allow you now to take certain problems that previously were just too expensive or too slow to do to have a decision model do this. And the way I think you should be thinking on this is that when you have code mode, what the harness can do is it can create a little workflow that can have decision points along the way where if you for instance have to crunch through 3,000 incoming customer support requests and you have to make decisions along the way, if this is being sent towards having an agent work on it towards requiring more feedback from a customer, you can have the agent create basically an ondemand workflow and then for every single if condition that it's doing, it's just going through F and then you can crunch through thousands of issues for very little time in no time in your coding agent where previously that would have taken you hours. Um so with this um I just want to leave this idea now that we can build everything maybe we should not um but we should really I think push towards much more enjoyable and um in many ways much more ambitious software than we have been doing before. Thank you.

[鼓掌]

英文原文

>> [applause]

软件工厂与工程演进

如果软件工程的未来不在于编写代码，而在于构建编写代码的系统呢？为了探讨 AI 驱动的软件工厂的兴起以及工程师角色的演进，请欢迎 Warp 创始人兼首席执行官 Zach Lloyd。

英文原文

>> What if the future of software engineering isn't about writing code, but building the systems that write it? to explore the rise of AI powered software factories and the evolving role of engineers. Please welcome Zach Lloyd, founder and CEO of Warp.

好的。

行。大家好。很高兴来到这里，今天我将聊聊软件工程的演变，以及它正在如何转变为“工厂工程”（factory engineering）。非常兴奋能分享这个话题。首先简单介绍一下我自己。

我做工程师已经很久了。我曾是 Google 的主任工程师（principal engineer），过去曾领导 Google Docs 套件的工程研发，帮助将该产品扩展到了数以亿计的用户。我现在是 Warp 的创始人兼 CEO。Warp 为企业构建了一个用于打造其自身内部软件工厂的平台，我今天会展示其中的一部分。此外，Warp 也以我们开源的 Agentic 终端而闻名，该项目拥有大约 65,000 颗 GitHub Star，被将近一百万名开发者所使用。我们的使命始终是帮助开发者更快地交付更好的软件；随着我们周围世界的改变，我们所做的事情能随之演进也是非常酷的。

我认为这是我们目前大多数人仍然所处的状态。跟我交流过的大多数开发者、大多数平台团队，他们所协作的个体人员都是在本地运行他们自己的 coding agent。也许他们一次同时运行多个 agent；可能是通过终端运行，也可能是通过 agentic 开发环境运行。

大家现场举手表决一下，实际上这里有多少人主要是以这种本地多线程的方式在工作的？

英文原文

Okay. All right. Hi everyone. Uh excited to be here and today I'm going to chat about the evolution of software factoring software engineering and how it's becoming factory engineering. So excited to do this. First a little bit about me. Uh I've been an engineer for a long time. I was a principal engineer at Google. I used to lead engineering on the Google Doc Suite. Uh helped scale that product to hundreds of millions of users. I'm currently the founder and CEO of Warp. Uh, Warp builds a platform uh for companies to build their own internal software factories. I'm going to show a little bit of that today. Um, Warp is also well known for our open-source Aentic terminal, has about 65,000 GitHub stars. Uh, it's used by almost a million developers. Our mission has always been helping developers ship better software more quickly, and it's it's it's cool to evolve what we're doing as the world changes around us. Um, this is where I think we're still mostly at. Uh, most developers that I talk to, most platform teams, um, they they are working with individuals who are running their own coding agents locally. Maybe they're running, uh, multiple of them at a time. Uh, could be through their terminal, it could be through an agentic development environment. Um just show of hands quickly actually how many people in here are mostly working in this fashion local multi-threading.

好的，依然是大多数人，但实际上也并非所有人。我认为这种情况将会改变，而且已经开始在改变了。其中的原因有这么几个：

其一，如果你把本地交互式提示作为与 coding agent 协作的主要方式，你其实受限于人类的吞吐量；如果每个 agent 都需要交互式地去提示它，你确实无法实现自动化。

还有第二个原因，我也经常从平台团队以及工程负责人那里听到：每个人都选择自己的本地配置、自己的模型、自己的 skill（技能插件），这种机制导致你非常难以确保从成本和生产力的角度真正高效地使用这些 agent。

最后，当每个人都在搭建和运行自己的本地 agent 时，还会带来很大的安全问题和治理问题。

因此，即将到来的东西——我粗略地称之为软件工厂，这对不同的人意味着不同的事情。我想知道现场在座的各位中，有多少人目前正在尝试构建、正在构建或者正在使用他们自己的内部软件工厂？举手看一下。

英文原文

Okay so still most people but actually not everyone um I think that this is going to change and is starting to change already. Uh the the reasons are there's a few of them. So one is like if you are doing local interactive prompting as your main way of working with coding agents um you're kind of limited by you know human throughput really like you cannot automate uh if every agent needs to be interactively prompted there's a second reason too that I hear a lot from platform teams um and from you know engineering leaders this mechanism of like everyone choosing their own local setup their own models their own skills uh it makes it very hard to actually make sure that you're using these agents efficiently uh from like a cost perspective and a productivity perspective. And then finally, there's like a you know big security problem and governance problem when everyone is setting up and running their own local agents. And so the thing that uh is coming uh I loosely call a software factory that means different things to different people. Um, I would well I'm curious for people in this room like how many folks are currently trying to build or building or using their own internal software factories? Show of hands.

了解。我想说此时此刻人数几乎比用交互式 agent 的还要多，这太酷了。

所以我要定义一下我心目中的软件工厂到底是什么，但我先讲讲我认为软件工厂不是什么。

我认为它不仅仅是一组松散的自动化集合。我不认为软件工厂是一群 AI 队友的集合。我不认为它是一组共享的技能和 MCP。而且我也认为软件工厂并不是个人 agent 群（agent swarms）。我认为这些虽然都很有用

英文原文

Got. So I would say that's almost more than the interactive agents at this point. That's super cool. Um, so I'm going to define what I think of as a software factory, but I'll start by saying what I think a software factory is not. Um, I think it's not just a loose set of automations. Uh, I don't think of a software factory as a collection of AI teammates. Uh, I don't think it's a shared set of skills and MCPs. And I also don't think software factories are personal agent swarms. I think these are all useful

软件工厂的定义与三大支柱

原语，但它们并不是我们应该追求的终态。

英文原文

primitives, but they're not the end state that we should be shooting for.

所以对我来说，我对软件工厂的定义非常具体。嗯，我会把它拆解开来讲，但我把它定义为：一个用于持续度量和改进云端编程智能体的代码定义闭环系统。这听起来有点拗口，但让我来拆解一下。如果你想构建一个软件工厂并真正扩大其规模，我相信你首先需要做的就是用声明式语言将其完整地编码出来。所以你需要说明，好，这些是我想要运行的所有编程智能体，这些是它们在工作进入该工厂时的触发方式，这是工作流，从分流到规格说明，无论你的工作流是什么，你都需要指定有哪些可用的 MCP、你的智能体在哪里运行，尽可能用类似于 Terraform 风格的一套声明式代码文件来定义你的工厂。我稍后会讲为什么这很重要。

英文原文

And so to me, the way that I define software factory is very specific. Um, and I'll unpack this, but I define it as a code defined closed loop system for continuously measuring and improving cloud coding agents. So that's a mouthful, but let me unpack it. So if you want to build a software factory and really scale it, I believe the first thing you need to do is fully encode it in a declarative language. So you need to say okay these are all of the coding agents that I want to run these are how they're triggered when work comes into this factory this is the workflow from is you know from triage to spec whatever your workflow is you need to specify what MCPs are available where your agents run really as much as you can in a Terraform style declarative uh you know set of code files that define your factory I'll talk about why this matters in a

第二部分是一个持续改进的闭环。这个闭环大致可以长成这样。这是很典型的一个。这实际上就是软件开发生命周期（SDLC）。其中你的想法来自人类，嗯，它们也可以来自自动化信号，然后你的智能体会承担一定量的工作，推动该问题经历各个阶段，从分流、实现、验证、监控等等。这里重要的部分实际上是这个闭环概念：你的智能体运行得越多，你就能把这个循环变得越高效。我稍后会谈谈具体怎么做。

英文原文

The second part of it is a continuous closed loop of improvement. And that closed loop can look something like this. This is the typical one. This is just really the SDLC. Uh where you have ideas that come in from people um uh they could come in from automated signals and then your agents do some amount of work um taking that issue through various stages from triage, implementation, verification, monitoring and so on. The important piece here is really this closed loop concept where the more that your agents run, the more efficient you can make this loop. And I'll talk about how you do that in a second.

第三个真正关键的方面，如果你要构建一个软件工厂，那就是一切都需要可度量。嗯，如果你在部署编程智能体时仅仅凭感觉，你就无法确信自己是否真的以最优成本获得了最高质量的产出。你无法确信自己是在最大化吞吐量。所以你确实需要一种度量思维和度量能力。

英文原文

The third really key aspect if you're building a software factory is that everything needs to be measurable. um if if you're just going on vibes in terms of how you deploy coding agents, you can't be confident that you're actually um getting the best highest quality outputs the at the best cost. You can't be confident you're maximizing throughput. So you really need a measurement mindset and measurement capabilities.

工厂工程学的新思维与新工具

因此，如果说我希望大家听完这个演讲能带走什么，那就是：对待编程智能体的方式以及思考它们的方式，在于这里出现了一个非常酷的、根本性的全新工程问题，即你如何利用我们在过去一年里获得的这个神奇的新原语——也就是这些智能智能体，以及你如何围绕它们构建一个系统来最有效地使用它们。对我来说，这并不像培育花园或遵循某个菜谱。

英文原文

And so if there's one thing that I'm hoping people walk away from this talk with, it's the idea that the way to work with coding agents and the way to think about them is that there there's a really cool fundamental new engineering problem, which is how do you take this magic new primitive that we all have that we've gotten the last year that you have these intelligent agents and how do you engineer a system around them to use them most effectively. To me, it's not about like growing a garden or following a recipe.

我们都是工程师。我们整个职业生涯都在学习如何以工程思维来解决问题。而当你把编程智能体视为一种新的工程原语时，你自然就会想要、也自然就会开始以某种特定的方式来构建使用它们的方法。这就是那种思维方式。

英文原文

We're all engineers. We've spent our careers learning how to solve problems from an engineering mindset. And when you think of coding agents as a new engineering primitive, then you naturally want to you'll naturally start to build a certain way of using them. That's the mindset.

所以，是的，稍微拆解一下这一点。这里既有持续度量和持续学习，但也有一套新的工具。

英文原文

So, yeah, just to unpack that a bit. So there's continual measurement and continual learning, but there's also a new set of tools.

当年我在 Google 学习如何成为一名更优秀的工程师时，嗯，我真的必须精通使用像调试器、性能分析器（profiler）和 Chrome 开发者工具这类的东西。基本上，手工作出软件有所有这些行业工具，它们现在也许没那么重要了。但它们确实曾把最优秀的工程师与普通的工程师区分开来。而在工厂工程的新世界中，有一套每个人都应该努力掌握的新工具。那些工具包括 eval（评估）。就是你能否建立起行之有效的自我改进循环？你能写基准测试吗？你能对你的工厂进行 A/B 测试吗？

英文原文

And so when I was learning how to become a better engineer at Google, um you know, I really had to get good at using things like debuggers and profilers and Chrome dev tools. And basically there were all of these tools of the trade for building software by hand that maybe are not as important now. But they really separated uh the the best engineers from the okay engineers. And in the new world of factory engineering there's a new set of tools that everyone should be trying to to master. And so those are things like eval. It's can you set up uh self-improvement loops that work? Can you write benchmarks? Can you AB test your factory?

这是一套非常重要的技能集。而掌握这套技能集，就是我认为的工厂工程。现在，这并不意味着这是你唯一要做的工程。你仍将进行产品工程，意味着作为工程师的你仍然应该对“你交付的产品是否真正解决了一个真实的用户或客户问题”负责。

英文原文

And that's a really important skill set. And that mastering that skill set is something that I I think of as like factory engineering. Now, that doesn't mean that that's the only kind of engineering you're going to be doing. You're still going to be doing product engineering, meaning you as an engineer should still feel responsible for, you know, does the product you ship actually solve a real user or customer problem?

但是最优秀的工程师——而且我想说基本上每个工程师——确实都需要学习这套新工具，即你如何改进系统，而不仅仅是构建产品。

英文原文

But the best engineers and I would say like every engineer really is going to need to learn this new tool set of how do you improve the system not just build the product

智能体系统的基础设施技术栈

那么这具体是什么样子的呢？嗯，如果你从第一性原理出发思考如何工程化一个智能体系统，我认为你想出的东西大致长成这样——不一定完全是这样，但嗯，这是你为了工程化你的智能体而将要使用的一种基础设施技术栈。在这最底层的是我所说的“代码化工厂”（factories as code）。我之前讲过这个。它是以代码格式定义你所有的编程智能体工作流。这样做的价值在于，你获得了从它拉出分支并测试不同工厂配置的能力。你可以回滚，而且关键是，它为自我改进开辟了道路，也就是编程智能体自身提出修改你工厂运作方式的建议的能力，从而让那个持续循环运转起来。

英文原文

and so what does that look like? Uh if you think from first principles about how do you engineer an agentic system I think you come up with something like this doesn't have to be exactly this but uh it's kind of an infrastructure stack that uh that you're going to want to use to engineer your agents. And so at the bottom of this is what I call factories as code. I talked about this earlier. It's defining all of your coding agent workflows in a in a code format. And the the value of that is that you get the ability to branch off it and test different factory configurations. You can roll back and then crucially it opens up the way for self-improvement which is the ability for coding agents themselves to suggest changes to how your factory works. to get that continual loop going.

在这之上是数据与上下文层。这也是至关重要的。因此，如果你在一家机构中大规模运行编程智能体，你绝对必须记录那些智能体产出的一切内容。也就是它们所有的对话、所有的工件、它们制作的视频、截图。嗯，这样做的原因在于，那是你后续用来改进工厂运转方式的原材料。

英文原文

Above that is a layer of data and context. This is also crucial. So if you are running coding agents at scale in an organization, you you really must be recording every everything that those agents are producing. So all of their conversations, all their artifacts, the videos that they make, the screenshots. Uh and the reason for doing that is that's the raw material that you will use to improve the way your factory works over time.

再往上是计算层。嗯，所以你需要具备沙箱环境。嗯，沙箱增加了安全性。它让你的工厂具有更强的可重复性。你的沙箱可能应该是多平台的。

英文原文

On top of that is a compute layer. Um so you need to have sandboxing. Uh sandboxing increases security. It makes your factory more repeatable. Your sandboxing should be probably multiplatform.

再往上是推理层。在工厂中，我对推理和模型的看法是，它们是一个可供测试的自由度。因此，无论你怎么构建你的工厂，它都应该支持多个模型。嗯，它还应该支持多个编程智能体，因为你知道云端代码可能在某些方面更好。Codex、pi、warp，这些 harness 各自具备不同的特性。你可能想要编写自己的编程智能体 harness，其中融入了你的一些组织上下文。所以你的平台应该支持这一点。

英文原文

On top of that is the inference layer. The way that I would think about inference and models in a factory is they are a degree of freedom to test. And so your factory, however you build it, should support multiple models. Um, it should also support multiple coding agents because you know cloud code might be better for some things. Codeex, pi, warp, these harnesses all have different properties. You might want to write your own coding agent harness that has some of your organizational context baked in. So your platform should support that.

在此之上，你需要一个编排层。这就是实际推动工作流经工厂的组件。在某些方面它是最不有趣的部分，但你需要它。然后最后，你的工厂应该完全以 API 和 MCP 为先，这样它不仅可以被人类使用，还可以被其他编程智能体使用，并且它可以延伸到实际开展工作的地方。所以你的工厂应该集成到像 Slack、Linear、Jira、GitHub 等工具中。

英文原文

On top of that, you need an orchestration layer. So this is what actually moves work through the factory. In some ways, the least interesting part, but you need that. And then finally, your factory should be all API and MCP first so that it can be used not just by uh you know people, but can be used by other coding agents and it can go to where the work is being done. So your factory should integrate into things like Slack, Linear, Jira, GitHub, and so on.

Warp 工厂功能演示

我打算在 Warp 工厂的背景下快速演示一下其中一些工具是什么样子的，但我认为我要展示的这些东西，在你构建自己的系统时也同样适用。

英文原文

I'm going to give a quick demo of what some of these tools look like in the context of warp factories, but this stuff I'm going to show I think applies also if you're building your own.

那么让我从这里开始。底层如我刚才提到的，是代码化工厂。所以我打算展示一些代码。它不会那么有趣，但如果你熟悉类似 Terraform 的东西，你就会明白这里发生的事情。你基本上需要一种声明式的方法来说明这就是我的工厂。我的工厂具有某些策略。它访问某些仓库。它有某些触发器。它具有某种蓝图，定义了流经它的工作的工作流。然后如果你放大来看，属于该工厂的所有其他这些事物也都有代码定义。

英文原文

So let me start here. So the at the foundation I mentioned earlier is factories in code. And so I'm gonna show some code. It's not going to be that interesting, but if you're familiar with something like Terraform, you will get what's going on here. You basically want a declarative way of saying this is my factory. My factory uh has certain policies. It accesses certain repos. It has certain triggers. It has a sort of blueprint that defines the workflow of work going through it. And then if you zoom out all of these other things that are part of the factory also have code definitions.

往上一层的第二点是，嗯，你在工厂里做的一切事情都应该在你的整个团队中被追踪。因此，将我在这里展示的内容理解为：这是一种类似于 Google 文档风格的实时视图，展示了 Warp 团队中跨所有工程师的每个编程智能体在所有时刻正在做的事情。因此我可以点击进去，我可以查看，好吧，我不知道 Maggie 在做什么，但我知道我可以看一下。她正在高级计划（high plan）上实现 Opus 55。她是在 Slack 中操作的。它有这个工件。我可以点进去查看这里的实际对话。我可以看到它生成了什么。所以我可以实时看到这一切，这很酷。但真正重要的在于你收集了所有这些信息，以便它可以用来，嗯，改进工厂。这就把我带到了第三点，在整个技术栈上更进一步，一旦你拥有了所有这些——

英文原文

The second thing moving up a layer is um the idea that uh uh sorry that everything that you do in the factory um should be tracked across your entire team. And so this think of think of what I'm showing here is this is a sort of Google Doc style live view of what every coding agent on Warp's team across all of our engineers are doing at all times. And so I could click in and I could see okay I don't know what Maggie is working on but I know I could look at it. She's implementing Opus 55 on the high plan. She's doing it from Slack. It has this artifact. I could look in and see the actual conversation here. I could see what it's generated. And so I could see this in real time which is cool. But the really important thing is that you gather all this information so that it can be used for um improving the factory. And so that brings me to the third you know the third thing moving up the stack here is once you have all of this

有了这些集中存储的信息，你就能非常轻松地了解这些编码智能体的工作状况，完全不需要在每个人的笔记本电脑上安装任何东西。你可以从传统产出指标的角度来考虑，比如你的 DORA 指标，像周期时间；但你也可以从自动化程度的角度来考虑，比如我实现了多大程度的自动化？每次 PR 需要多少次人工交互？你的吞吐量是多少？你应该能在这里非常清楚地看到关于成本之类的数据。再次强调，把这些数据呈现出来至关重要，如果没有这些数据，你就像是在盲飞。

英文原文

information that's being stored centrally you get the ability very easily uh without doing any like install stuff on everybody's laptop anything like that to to understand how these coding agents are working and you can think of that in terms of like traditional output metrics like your Dora metrics like okay cycle time but you can also think of that in terms of like how automated ated am I like how many human interactions per PR what's your throughput you get um you should be able to see data very clearly here on like what's your cost and again surfacing this if if you don't have this data you you're kind of flying uh oh you're kind of kind of flying blind um so this is really important

然后是下一层数据，这同样非常重要，也就是我之前提到的能力：设置跨所有编码智能体运行会话的在线评估（online evals）。我这么说是什么意思呢？评估就是，在这些编码智能体会话运行并记录其产出时，你希望能够让另一个智能体回过头来审视它们，并从某个可能反映它们工作表现的维度进行打分。

英文原文

and then there's this uh next layer of data which is also really important that I mentioned earlier which is the ability to set up online evals that run across all your coding agent sessions. And so what I mean by this, an eval is just like as these coding agent sessions are running and they're recording their output, you want uh the ability to to go back and have another agent look at them and score them on some dimension that might indicate how well they're doing.

具体来说，比如我们这里使用的一个评估评分器，它是用来查找冗余测试的。这是一个很好的例子。编码智能体常见的一种失效模式就是，有时它们会编写不必要的测试。我们希望能捕获到这种情况。这就是该评估所做的事情。这样我们就能看到：好吧，当它编写了过多测试时，为什么会这么做？软件工厂中是否存在某种我们可以更改以进行改进的机制？

英文原文

And to make that more concrete, so it's like we have an eval a scoreer here that we use that is looking for redundant tests. This is a good example. And so one common failure mode of coding agents is that sometimes they will write tests that are unnecessary. We want to catch that. And that's what this eval does. We want the ability then to see okay when it's writing too many tests, why is it doing that? And is there a mechanism that we could change in the factory for improving that?

所以再次简单总结一下：你需要一种记录所有运行的方法。当这些运行执行时——刚才我没有展示——你希望有一种方式来指定它们具体运行的循环是什么。你希望能够看到它们处于该循环的哪个环节。随着运行的进行，你希望能够收集它们的数据。一旦收集了数据，你就能设置这些在线评估。接着，这些在线评估能让你了解：好吧，哪里存在改进的空间？

英文原文

And so again, just to kind of recap here, uh you want a way of recording all the runs. When these runs run, I didn't show this, but all of the runs are, you know, you want the a way of specifying um what is the sort of like actual loop that they're running through. You want to be able to see where they are in that loop. As the runs uh work, you want to be able to gather their data. Once you gather their data, you want to be able to set up these online evals. And then these online eval give you a sense of like, okay, where are their opportunities for improvement.

一旦落实了这一点，你就可以进行一项我们称之为自我改进（self-improvement）的工作。与其让人类去查看这些评分器的输出，不如让智能体来查看输出。理解自我改进的方式是：它查看评分输出，发现我们看到了若干次失败的运行，接着尝试在这些失败中寻找模式并提出可能的解决方案，然后该解决方案以针对你工厂定义的 PR 形式呈现。

英文原文

Once you have that in place, you can do something that we call self-improvement, which is rather than having humans look at the outputs of these scores, you have agents look at the output. And so the way to think about self-improvement is like it looks at the score output. It says okay we saw some number of failed runs um let's try to look for patterns in those failures and suggest a possible solution and then the form that that solution takes is as a PR against your factory definition

将工厂定义为代码并能够使用编码智能体的绝妙之处就在于，你可以让编码智能体诊断它们内部工作方式存在的问题，并向其提出修复建议。

英文原文

and so this is like the awesome thing about having your factory defined as code and being able to use coding agents is that you You can then have the coding agents diagnose problems with how they're working internally and suggest fixes to them.

基准测试与总结

我要指出的最后一个工具，正如我之前提到的，是创建和运行基准测试（benchmarks）的能力。这也是必不可少的。如果你关注网上的讨论，经常会看到类似“这个模型最好”、“这是新的最佳技能”、“这是最佳 harness”这样的说法。真正回答这些问题的唯一方法，就是在你自己的数据上进行测试。因此，你应该建立起某种机制，基本上可以利用你的真实数据创建一个基准测试集，然后重放这些任务……哎呀，我这里翻错页了……重放这些任务针对不同的配置进行测试，看看如果你更改某个模型、更改某个 harness 或更改工厂配置，它们会有怎样的表现。

英文原文

The final tool that I'm going to point out that I mentioned earlier is the ability to create and run benchmarks. This is also essential. So if you're following discourse online, you'll often see things like, okay, this is the best model. This is the new best skill. This is the best harness. The only way to answer those questions for real is to test on your own data. And so you should have in place some mechanism where you can basically create a benchmark set out of your real data. Um, and then uh replay those tasks. Oops, I'm on the wrong one here. And replay those tasks against different configurations to see how they would have done if you had varied some model or varied some harness or varied your factory configuration.

为了说明清楚，这看起来就像新模型发布时你在 SWE-bench 或 TerminalBench 中看到的那样。它在成本和质量方面展示了不同模型的权衡取舍。你希望处于左上角的象限，但这是基于你自己的真实数据。你不仅可以测试模型，还应该能够测试：嘿，如果我去掉这些技能，我的工厂表现是会更好还是更差？这些就是其中的一部分工具。

英文原文

So just to make this clear, this is something that looks like what you might get uh when a a new model drops like in SweetBench or Terminal Bench. It's showing the tradeoffs of different models here in terms of cost and quality. You want to be in this upper lefth hand quadrant, but it's on your own actual data. And you can you can test not just models but also you want to be able to test hey if I got rid of these skills does my factory perform better or worse. So these are some of the tools.

回到核心概念上来，这套方法在我们这里运行得很好。我们成功利用这些工具大幅提升了吞吐量并降低了成本。我们一直在持续做这件事，因为成本对我们来说真的很重要——我们在建立性能基线，我们在打分，我们在做基准测试，我们在测试，我们在做工厂工程化。

英文原文

Um and just to return to this big concept like this has worked well for us. We've managed to use these tools to drastically improve our throughput and lower our cost. We we are just constantly doing this because this cost really matters to us where we're getting a performance baseline. We're scoring, we're benchmarking, we're testing, and we're doing factory engineering.

这让我回到了我想再次强调的最后一点：工程化你的智能体。它是一个全新的工程原语。我们都是工程师。思考一下如何最大程度地发挥它们的价值，并为此构建系统。我认为这非常令人兴奋。我知道现在关于作为一名工程师意味着什么有很多焦虑。我知道我们现在不怎么写代码了，但我仍然在使用我整个职业生涯所学到的全部工程技能。所以对我而言，这依然是一个非常令人兴奋的时代。如果你对更多关于工厂的内容感兴趣，可以在这里查看。感谢大家来听我的演讲。

英文原文

And so that brings me back to this last point that I just want to emphasize again. Engineer your agents. It's a new engineering primitive. We're all engineers. Think about how you can get the most out of them and build the system for that. I think this is very exciting. Um, I know there's a lot of like angst around what it means to be an engineer right now. I know we're not writing code, but I'm still using all the engineering skills that I spent my whole career learning. And so to me, it's still a really exciting time. Uh, if you're interested in more factories, you can check it out here. Thank you all for coming to the talk.

[掌声]

英文原文

[applause]

Brandon Thomas 演讲：Artcraft 与开源的未来

当 AI 智能体只需数周就能构建出完整的创意类应用程序时，会发生什么？我们的下一位演讲者是 Artcraft 的创始人，这是一个探索 AI 赋能软件开发可能性的项目。女士们，先生们，请热烈欢迎 Artcraft 首席执行官 Brandon Thomas。

英文原文

What happens when AI agents can build entire creative applications in just weeks? Our next speaker is the founder of Artcraft, a project exploring what's possible with AI powered software development. Ladies and gentlemen, please welcome Brandon Thomas, CEO of Artcraft.

大家好，感谢大家来听我的演讲。我是昨晚深夜才把这些内容整理出来的。在这个为成千上万开源开发者排忧解难的项目火爆出圈之后，我一直处于一阵旋风之中。有风险投资人主动接触我们，我一直在开会。事情发生以来的这一个月简直是连轴转、通宵达旦。如果你还不了解的话，我差不多克隆了 Adobe Photoshop 和一堆其他应用，然后这事在网上爆火。我的手机、我的收件箱，一切都被信息彻底挤爆了。所以我对这场演讲的结构表示抱歉，这是我以最快速度凑出来的，但我有很多非常令人兴奋的内容要讲，我认为这探讨的不仅是行业的未来，更是整个技术的未来。

英文原文

Hey everybody, thanks for attending my talk. I had to put this together late last night. I've been in a whirlwind after this project went viral of unblocking thousands of open- source developers. Uh we've had VCs approaching us. I've been in constant meetings. Like this has been a sleepless month of activity since this happened. So if you're not familiar, uh I kind of cloned Adobe Photoshop and a bunch of other apps and that went super viral. My phone, my inbox, everything has just been absolutely lit up. And so I I I apologize for the structure of the the the the talk. This was thrown together as quickly as I can, but I've got a lot of really exciting stuff to talk about and I think it talks about the future of not only the industry, but just technology as a whole.

那么，我们打算把整个世界都开源。为什么不呢？这是我作为一名开发者一直想做的事。这也是我们开源社区很多人都感兴趣的事情。开源支撑着许多我们熟知的技术，但它从未走在最前沿，比如 Linux 一直就像二等公民。我们被卷入了一大堆自己并没有真正所有权的平台中，而可以说 AI 给了我们一个把网络夺回来的机会。

英文原文

So um we're going to open source the entire world. Like why not? It's it's something that I've always wanted to do as a developer. That's something that uh a lot of us in the open source community are interested in. uh open source powers uh uh a lot of the you know technology that we're familiar with but it's never been at the forefront like Linux has always been like a secondass citizen. We've been pulled into a whole bunch of platforms where we don't really have ownership and AI kind of gives us a chance to take the web back if you will. So um

我可能得手动翻页了。好的。

英文原文

I might have to do this manually. Cool.

自我介绍一下。我做工程师大约有 15 年了。我在 Square 工作了 8 年，从事重量级的基础架构开发，包括在线支付、认证技术栈，基本上任何可能导致公司瘫痪的系统，差不多都由我的团队负责。那是值守寻呼机的 8 年，做了大量非常核心的基础架构工作。然后我花了 7 年时间做初创公司，那完全是另一个世界：有很大的风险，要不断了解市场需求，不断纠正自己的假设。

英文原文

So a little bit about myself. I've been an engineer for about 15 years. I spent eight years at Square doing uh heavy-hitting Enfra, so online payments, um the uh the authentication stack, basically anything that could take down the company my team was kind of in charge of. And it was, you know, uh eight years of carrying the pager, a lot of really heavy-hitting infrastructure work. Um and then I've spent seven years doing startups, and that's a whole different world. It's a lot of risk, a lot of learning like what the market needs, a lot of uh correcting your own assumptions.

而且我一直……我常驻亚特兰大，这是一个电影大州，我非常热衷于艺术。大学毕业后，我拍过很多电影，制造过在摩天大楼侧面打游戏用的激光投影仪等等。这些正是我们的初创公司或者我创办的初创公司所迎合的客户群体：创意工作者、艺术家、电影制作人等等。

英文原文

And I've also been um I'm based in Atlanta. It's a big film film state. I've been very big in the arts. So out of college, made lots of films, built laser projectors that play video games on the side of skyscrapers, that kind of thing. And those are the sort of uh customers that our startups or the startups that I built have have catered to, creatives, artists, filmmakers, and so forth.

我想把大家带回过去，回到网络和互联网的最早时期，90 年代和 2000 年代初期。对于我们千禧一代来说，会觉得自己老了。而对于所有更年轻的人来说，这只是让我为大家铺垫一下当年的互联网是什么样子的。当时我们有一批完全开放的聊天应用。比如，IRC 就像 Discord，但你可以启动自己的……

英文原文

Um, I want to transport everybody back in time, back to the very beginnings of the web and the internet, the '9s, the uh the the early 2000s. So, for us millennials, we're going to feel old. And for um everybody that's that's younger, this is just let me kind of set the stage for what the internet was like. Um, we had a bunch of chat applications that were completely open. So, IRC is like Discord, but you can spin up your own

早期网络的开放生态与平台化转变

真正属于你自己的服务器。你可以使用任何程序来自动处理它们，并毫无规则限制地向它们查询数据。所有的商业即时通讯应用基本上都不介意你使用第三方工具来连接。每个网站都有数据导出功能，都有 API。你真的可以用 Reddit、Flickr 和 Twitter 构建出非常酷的混搭应用（mashups）。在 Hacker News 出现之前，这些东西在 Slashdot 上就非常流行。然后，在更现代的时期，这些发展放缓到了慢得多的节奏，因为 API 被移除了。

英文原文

servers that you actually own. you can use whatever programs to automate them and interrogate them without rules. Uh all of the commercial uh messaging apps were basically they didn't care if you use thirdparty tools to connect to them. Uh every website had data exports. They had APIs. You could literally build really cool mashups with Reddit and Flickr and Twitter. And these were like trending on Slash before Hacker News was a thing. And then you know and and in the more modern times uh these kind of trickled down to a much slower pace because the APIs got got removed.

当时有大量的文件共享。不仅是文件共享，你还可以真正购买 MP3，在网络上所做的每件事中感受到一种所有权。而现在更多的是许可。当时有如此多的开放创造力，主要是因为互联网上的人都是技术人员，都是早期采用者。所以玩 Flash 的孩子们制作了各种动画，还有基于 PHP、phpBB 和 Invision 的分散社区，当时涌现出大量的创造力。维基百科也是在这种精神氛围中创立的，而在当今时代和技术的当前形态中，我们已经看不到这类东西了。当时的网站具有极高的可定制性，比如当时的 MySpace，你确实可以修改 HTML 和 CSS，让它看起来完全符合你的想法。因此，与我们今天所拥有的相比，那是一个完全不同、截然陌生的时代。

英文原文

Uh there was a bunch of file sharing. So not not just file sharing. You could actually buy MP3s and feel a sense of ownership and and everything that you were doing on the web. And now it's more licensing. Um there was so much open creativity because largely the people on the internet were technical. They were early adopters. So Flash kids were making all kinds of animations. Uh there were diffuse communities with PHP uh uh PHPVB and Envision and just just a ton of creativity that was happening at the time. Um Wikipedia got started sort of in this ethos and we don't really see this kind of stuff anymore in the current era and incarnation of tech. Um websites were highly customizable like MySpace at the time. you could actually change the HTML and CSS and make it look however you want. So, this was a totally different alien time compared to what we have today.

当 Web 2.0 出现时，宽带、智能手机把所有人都吸纳了进来。于是，技术人员在某种程度上被边缘化了。这并不是一件坏事，比如我们需要让我们的父母和祖父母能够使用这些产品。但是我们削去了棱角，拿掉了所有的技术组件，让一切变得容易让人们适应，因为他们无法从技术的立足点去做出贡献。他们不想去拆解东西，不知道 API 是什么。因此平台和原……比如协议被平台所取代，它们竭尽所能地攫取尽可能多的用户，早期那种分散的、实验性的网络开始消失。

英文原文

Um, when web 2.0 happened, broadband, smartphones, it brought everybody into the fold. So, the technical people got sort of pushed to the sidelines. And it's not a bad thing like we needed our, you know, our parents and our grandparents to be able to use these products. Uh, but we sort of we rounded the edges on things. We took away all the technical pieces. We made everything easy for people to adapt because they couldn't contribute from a technical footing. They didn't want to unpackage things. They didn't know what APIs were. And so the platforms and the the the proto like protocols got replaced by platforms and uh they sort of grabbed as many users as they could and sort of the diffuse experimental early web started to disappear.

闭源垄断与寻租模式的加剧

API 从我们手中被夺走了，然后我们开始看到分发渠道被大服务商把守。比如我们从来没有过一个可以像在桌面上那样自由进行分发的手机生态系统。随着初创公司开始赢得市场份额，东西开始变得越来越昂贵。他们意识到自己拥有定价权，突然之间，一旦你订阅了所有的流媒体服务，你的 Netflix 实际上就变成了有线电视。

英文原文

the APIs got taken away from us and then we started seeing distribution being gatekept by the big providers like we never once had a phone ecosystem where we could freely distribute to like like we could on the desktop. Um things started to cost more and more as as startups began to like win market share. They realized that they had pricing power and uh suddenly your Netflix actually became cable once you started subscribing to all the streaming services.

然后对于那些或许已经触及增长极限的公司，它们开始变得极具攫取性。它们开始想出极其离谱的套路。让我中招的一个例子是 Adobe，它有按月付费方案，而且他们不会告诉你这一点，而是写在非常小的细则里：如果你在 12 个月或 18 个月之前取消月付方案，他们实际上会向你收取高达约 200 美元的退订费。如果你是一年参加一次电影节、制作一部短片，然后突然之间为了一款只在一个周末使用过的应用程序背上一张额外的 200 美元账单，这真的非常麻烦。这坑了很多人。曾经非常非常充满希望、开放和自由的网络，变成了这个被占有、被租用、寻租的世界，这真令人沮丧。在 90 年代和 2000 年代它曾经是一个非常特别的地方，而现在被榨取的价值越来越多。常用的一个词是“平台恶堕”（enshittification），我认为这恰如其分地描述了所发生的一切。

英文原文

And then they started to get very extractive for companies that maybe maybe hit the limits of their growth. They started coming up with crazy crazy gotchas. So the one that got me is Adobe has this uh you can subscribe to a monthly plan and they don't tell you this. It's in like the very small print. If you cancel your monthly plan before the 12 months or 18 months, they actually charge you up to like $200 to unsubscribe, which is really inconvenient if you're going to a film festival like once a year, making a short film, and then suddenly you've got an extra $200 bill for an application that you were just using over a weekend. And that's that's hit so many people. And it's really frustrating uh that the web that was once really really promising and open and free uh has turned into this uh extractive owned rented rent seeeking world. Um so it it used to be a very special place in the 90s and the 2000s and more and more has been extracted. Uh this is a term that's commonly used um in shitification. And I think that aptly describes everything that happened.

以开源与 AI 重塑创作工具

我们公司创立时的宗旨就是“把一切都构建成开源”。比如我们在 AI 领域构建的应用程序是 100% 开源的，不仅是桌面应用，实际的基础设施也是。Artcraft 最初的定位有点像面向专业人士的 Higgsfield。我认识并与之一起长大的所有电影制作人，我们是在为他们做一款可供使用的产品，而不是像 Higgsfield、OpenArt 等那样采用以 UGC 为中心的方式。这使我们能够为他们提供一个运行在他们机器上的桌面工具，他们生成的所有文件都存放在紧挨着他们编辑器的地方。

英文原文

Um, our company started with the ethos of let's build everything open source. Like the applications that we built in the AI space were um, 100% open. Not only the desktop apps, but the actual infrastructure as well. So um, Artcraft originally started as sort of like a professionally minded Higsfield. So all of the the filmmakers that I know and that I've I've I've grown up with, we were making a product for them to use rather than a UGC centered approach uh like like Higsfield or or uh Open Art or or so forth have. And that enabled us to give them a desktop tool that lived on their machine where all the files that they were generating lived right next to their editor.

我们观察到的一件事是——我的顾问们觉得我简直疯了，认为艺术家不懂什么是开源，他们不需要开源，不谈论开源，开源并不会真正被艺术家使用——我们居然观察到艺术家们在我们提供给他们的软件之上，用 vibe coding 编写他们自己的工作流。这对我们所有人来说都是崭新的现象。比如有一些微短剧工作室在为一些头部的微短剧初创公司做订单履约，他们每个月要完成 3 万到 5 万美元的履约额，团队规模很大。市面上没有任何产品能满足他们的需求，没有人会特意去开研讨会并专门针对他们所需的交付模式进行构建。然而他们内部有人懂 Claude Code，他们可以调用 API，可以生成一个那样的界面。于是我们开始尝试完全开源并提供给我们的客户，他们对此反应非常积极。

英文原文

Uh and one of the things that we observed um and my advisers thought I was absolutely crazy that artists, you know, they don't know what open source is. They don't they don't need it. they don't talk about it. You know, isn't really used by artists. Uh we actually observed artists vibe coding their own workflows on top of the software that we gave them. And that is something that like was brand new to all of us. Like uh there were micro drama studios that were doing order fulfillment for some of the top uh micro drama startups. Uh they would do 30 to 50 grand of fulfillment every single month and they had big teams. Uh and no product out there fits their needs. Like nobody's going to go and workshop and build that specifically for the type of delivery that they need. Yet they in-house have people that know cloud code and they can take the APIs and they can generate an interface that's like that. So we started experimenting with completely open source giving it to our customers. Um and they really responded to that.

然后我们就想，如果我们能让一切重回过去呢？如果所有基础设施都是开源的，如果他们想要利用的所有应用形态都是开源的，那么他们消费和构建的所有这些东西都会友好得多，并赋予他们大得多的创造能力。比如他们使用 Premiere，使用 Adobe，我们能做些什么来给他们提供一些东西，让他们能够真正剥离所有用不到的表面积，把应用程序限制在他们的具体工作流中？如果他们团队中负责该工作流特定部分的每位不同艺术家都能这样做五次或十次，那会是什么样？因此，我们想要扭转 Web 2.0 时期发生的一切——当时一切都被所有权化、整合与把守——把应用程序重新还给人们，而 AI 确实就是我们做到这一点的手段。

英文原文

Um, and then we thought, what if we could turn back time on everything? Like all of these things that they're consuming and building would be much more friendly and and and give them way more capacity to create if all of the infrastructure was open source. If all of the application shapes that they wanted to leverage were open source. Like they use Premiere, they use Adobe. like what could what what could we do to give them something that they could actually take away all of the surface area that they don't use and constrain the apps to their specific workflow? What would it look like if they could do that, you know, five or 10 times for each different artist on their team that's doing a particular part of that workflow? And so we wanted to kind of undo what had happened during web 2.0 0 when everything became owned and and consolidated and and uh uh gatekept and just give people back their applications and AI really is the means for us to do that.

用 Agent 模式复刻 Adobe 系列

于是我们尝试对 Adobe Creative Suite 下手。我们首先把目光投向了 Photoshop。我们基本上通过提示词输入：这是一套照片编辑应用的基本形态，你已经摄入并训练过了所有关于 Photoshop 是什么的教程，这些都在你的训练数据中。我们写了一个非常非常庞大的提示词，要求构建出这款应用程序；将 UI/UX 与其分离；让所有东西都兼容 MCP 和 CLI；为每一个工具及其运作方式制定规划；讨论数据结构和文件格式导入器。我们基本上就是构建了这些大型提示词。在 24 小时内，我们看到了一个对 Photoshop 相当不错的复刻近似。这让我们震撼不已。它功能完备，对我们来说非常熟悉，启动速度比原版 Photoshop 快了 10 倍。

英文原文

So we took a stab at the Adobe Creative Suite. Uh we looked at Photoshop first. We basically prompted you know um this is the basic shape of a photo editing app. You you've consumed and trained on all of the tutorials about what Photoshop is. That's in your training data. uh we we laid out a large large uh prompt to say let's let's build out the the application. Let's separate the UIUX from that. Let's make everything MCP and CLI compatible. Let's build out plans for every single tool, how all of those work. Let's talk about the data structures, the file format importers. We basically built these large prompts. And within 24 hours, we were looking at a pretty good approximation of what Photoshop was. And it it kind of blew our minds away. Uh it was functional. It was very familiar to us. It started 10 times faster than the original Photoshop.

现在，我们并没有实际覆盖 Photoshop 的全部表面积，还有很多工作要做。我们仍未达到 100% 的对等水平，但我们正在对照需要完成的目标来衡量自己，并且越来越接近。我们之前达到了 50%，刚才跨过了 60%。假以时日，遵循帕累托法则，我们能够达到 99.99% 的对等（parody）。我稍微跑个题，我们的团队讨论过是现在就进一步拓展，还是先追求对等。目前我们只是希望满足所有使用 Adobe 的艺术家的肌肉记忆。因此我们直接朝着 Adobe 那样的界面与外观体验去努力。假以时日，我们将构建全新的应用，拥有更具 Agent 特征（agentic）的全新界面。但我们这样做也是作为一种检验方式，证明我们完全通过 Agent 方式确实做出了实质性的成果。于是我们做了 Photoshop，基本上做了他们所有的应用。我们做了 InDesign，我们做了

英文原文

Now, we didn't like actually cover the entire surface area of Photoshop. There's a lot of work. We're still not at 100% par, but we're measuring ourselves against what has what needs to be done and we're getting closer and closer. We were at 50%, we've just passed 60%. And in the fullness of time, like Pareto principle, we could reach 99.99% parody. Um I'm going to take a little bit of a sort of a diet trap. Our team has talked about whether we take this further now or whether we just go for parody and right now we're just we want to have we want to fulfill the muscle uh memory of all the artists that use Adobe. So we are going for Adobe like their sort of interface and look and feel directly in the fullness of time. We will build net new apps that have new interfaces that are more agentic. Um but we wanted to do that also as sort of like a proving function that we have actually accomplished something real completely agentically. Uh so we did Photoshop, we did um basically all of their apps. We did in design, we did

项目爆发与社区反响

Premiere……嗯，还有……这个，我完全没想到它会爆火。我当时甚至都没在 Hacker News 上发布。它先是在 Reddit 上传开，然后大概过了两天，《纽约时报》就联系我想进行采访；我们的 Discord 成员从 3,000 人一路激增，现在的成员人数甚至超过了 Jev、Suno、Higgsfield。我至今都难以消化这一切究竟是怎么发生的。它触动了人们的神经。大家某种程度上觉得自己被 Adobe 和其他科技公司慢待甚至伤害了，而我们所构建的东西真的引起了他们的共鸣，这在某种程度上凝聚成了一种继续构建更多东西的使命感。

英文原文

Premiere

um

and

uh

this

I

I

did

not

expect

this

to

go

viral.

I

was

withholding

posting

it

on

Hacker

News.

It

got

out

on

Reddit

and

then

like

2

days

later

like

the

like

New

York

Times

was

asking

me

for

an

interview

and

our

Discord

went

from

3,000

people

to

it's

now

got

more

people

in

our

Discord

than

Jev

Sunno

Higsfield.

It

I

I'm

still

struggling

to

bear

with

what

has

happened.

This

was

a

it

struck

a

nerve

with

people.

Uh

they've

felt

sort

of

maligned

by

Adobe

and

other

tech

companies

and

uh

what

we've

built

has

has

really

resonated

with

them

and

it's

sort

of

built

up

sort

of

this

mission

to

build

more.

所以，所有那些一直以来都在抵制 AI 的反 AI 艺术家们，居然都转变了态度，表示：“其实我们很喜欢你们在做的事情。”他们开始使用这些应用，向我们提出功能需求并提交 bug 报告，我们也积极与他们互动。而且我认为我们将能够教会他们：什么是 MCP，如何利用智能体（agentically）进行创作。也许我们能逐渐摆脱那些臃肿的界面，带给他们更快、更流畅、能让他们工作效率更高的工具。

英文原文

Uh,

so

all

of

the

anti-AII

artists

who

have

been

pushing

back

against

AI

all

of

this

time

have,

you

know,

they've

sort

of

turned

around

like,

we

actually

love

what

you're

doing.

They've

started

using

these

apps.

They've

started

giving

us

like

feature

requests

and

bug

reports

and

we're

engaging

them.

And

I

think

we're

going

to

be able

to

teach

them

this

is

what

an

MCP

is.

This

is

how

you

agentically

create

things.

And

maybe

we'll

start

pulling

away

some

of

these

bloated

interfaces

and

giving

them

things

that

are

faster,

more

fluid,

and

let

them

do

their

work

faster.

应用生态扩展

我们并没有止步于那几款应用，而是做了各种各样的探索。我们做了 CAD，做了 Flash，还做了一批数字音频工作站（DAW）。不仅仅是我们在做，社区也在做这些事情。而且在能做的事情上，几乎没有任何限制。我们重构了整个微软——呃，也就是微软 Office 套件里的核心应用，还有 Google Docs。很多公司、大学以及各行各业的人都找上门来想使用这些东西，我们基本上在竭尽全力满足所有这些不同的诉求。它们正被运行在以前从未支持过的 Linux 上。

英文原文

Um,

so

we

didn't

stop

with

those

apps.

We've

done

all

kinds

of

stuff.

We've

done

CAD.

We've

done

um

uh

uh

uh

uh

Flash.

We've

done

uh

a

bunch

of

digital

audio

workstations.

It's

not

just

us,

it's

the

community

doing

these

things

as

well.

And

there's

really

no

limit

in

terms

of

what

can

be

done.

Uh

so

we've

done

the

entire

Microsoft

well

the

core

apps

within

the

Microsoft

Office

suite.

There's

been

Google

Docs.

We

have

uh

companies

and

universities

and

all

sorts

of

people

approaching

us

wanting

to

use

these

things

and

we're

we're

basically

trying

to

fulfill

all

of

these

different

asks.

Um

they're

being

used

on

Linux

where

they've

never

been

used

before.

PDF Craft 可能是其中最出色的应用，你们现在都应该去下载一个。以后你们再也不需要用 Acrobat 了。这个应用真的非常快、非常顺滑。

[掌声]

它能在 Linux 上跑，能在 NetBSD 上跑，非常出色。这款已经完全可用了，大家尽管去下载体验。

英文原文

Um

PDF

Craft

is

probably

the

best

app.

You

should

all

go

download

it

right

now.

You'll

never

have

to

use

Acrobat

again.

This

is

like

really

fast,

really

smooth.

[applause]

It

works

on

Linux.

It

works

on

NetBSD.

It's

brilliant.

Like

this

one's

ready

to

go.

So

go

ahead

and

get

this

one.

我们还做了 Figma。实际上我们连服务端的组件也做出来了。团队基本上可以直接在文档上进行协同编辑，它具备类似 Raft、Paxos 这种多用户一致性架构。所以我们现在也开始涉足基础设施领域了。整个基础设施和数据模型，所有的一切全部都是开源的，我们不保留任何专有代码。

英文原文

Um

we

did

Figma.

We've

actually

done

the

server

component

for

this

too.

And

you

can

basically

have

your

team

collaborate

on

on

documents.

It's

got

like

the

raft,

Paxos,

like

multi-user

setup.

So

we're

starting

to

go

into

infrastructure

too.

And

the

entire

infrastructure

and

data

model,

all

of

it

is

open

source.

We're

not

keeping

any

of

it

proprietary.

有一个由工程师主导的社区正在开发 SolidCraft。他们实际上搞了一个开源的工程内核来管理这一切，确实非常扎实。借助 MCP，你用 SolidCraft 就能非常迅速地完成 3D 打印项目。

英文原文

Um

there's

a

community

that's

working

on

SolidCraft

that's

led

by

engineers.

They've

actually

got

uh

an

open

engineering

kernel

that

manages

all

of

this.

Uh

and

it's

it's

really

solid.

Like

the

MCP,

you

can

actually

do

3D

printing

projects

really

quickly

with

SolidCraft.

开源规模与跨平台能力

我们经历了太多的事情。我们已经获得了超过 3,000——30 万个 GitHub star，有 4,000 名不同的人在为这个项目贡献代码，势头还在加速。它真的已经演变成了一场运动（如果你愿意这么称呼的话），面对如此庞大的贡献者涌入，我们正在努力适应。我得加快点节奏了，因为我前面有点拖慢了进度。但目前我们每天的下载量大约有 170 万次，而且还在不断加速。

英文原文

Um

we've

had

so

so

much

happen.

We've

got

over

3,000

300,000

uh

GitHub

stars.

4,000

different

people

have

been

contributing

to

this

project

and

it's

picking

up.

It's

it's

it's

actually

become

sort

of

a

movement,

if

you

will,

and

we're

we're

trying

to

adapt

to

how

much

influx

of

of

people

trying

to

contribute

to

this.

I'm

going

to

pick

up

the

pace

because

I've

been

kind

of

lagging

on

this,

but

um

uh

we're

getting

about

1.7

million

downloads

a

day,

and

this

is

accelerating.

我们有一个桌面启动器应用，你基本上可以在上面管理所有的东西。它支持所有操作系统。我们有四分之一的用户在 Linux 上，这非常令人着迷。一直以来 Linux 平台受到的支持都非常匮乏，而现在他们拥有了用于创意和办公工作的超高质量专业应用，这真的很棒。我们非常尊重和喜爱过去在那方面所做的一切努力，但大家对于能用上以前接触不到的应用感到无比兴奋。

英文原文

Uh

we've

got

a

launcher

app

that

puts

us

on

the

desktop

and

like

you

can

you

can

basically

manage

everything

here.

It

works

on

all

operating

systems.

Um

a

quarter

of

our

users

are

on

Linux,

which

is

really

fascinating.

like

Linux

has

been

historically

really

really

underserved

and

now

they

have

super

high

quality

professional

apps

for

creative

and

and

uh

office

work

which

is

it's

really

nice.

So

um

we've

you

know

we

we

love

and

respect

uh

all

of

the

work

that's

been

done

there

in

the

past

but

people

are

really

excited

to

use

the

apps

that

they

didn't

have

access

to.

所有这些东西都是原生支持 MCP 和 CLI 的一等公民。你完全可以用命令来操控它们，能亲眼看着它们直接在你面前构建和组装创意项目。此外，它们还被本地化翻译成了以前 Adobe 或微软官方从未支持过的语言。比如曾有人告诉我们 InDesign 不支持阿拉伯语。我们还接盘支持了那些超出官方生命周期的应用。比如 Adobe Animate 走向寿命终结（EOL）或者基本处于维持状态，我们把它接了过来。基本上有动画工作室跑来跟我们说：“我们工作全靠这个，这至关重要，请务必支持它。”因此很多非常神奇的事情正在发生，我们正把这些工具带到比以往更多的地方。

英文原文

Um

all

of

these

things

are

MCP

and

CLI

like

first

class.

You

can

totally

command

these

and

you

can

watch

them

build

and

assemble

creative

projects

right

in

front

of

you.

Um,

they've

been

localized

into

languages

that

were

never

like

officially

supported

by

Adobe

or

Microsoft.

Like

we

were

told

that

Arabic

was

not

supported

in

Inesign.

We've

also

supported

these

apps

past

their

support

life

cycle.

Like

Adobe

Animate

was

being

uh

uh

end

of

life

or

it

was

being

like

put

on

life

support.

We've

we've

picked

that

up.

We've

had

basically

animation

studios

come

to

us

saying

we

use

this

for

our

jobs.

Like

this

is

important.

Please

please

support

this.

So

a

lot

of

a

lot

of

really

fascinating

things

are

happening

where

we're

getting

this

into

more

places

than

they've

ever

been

before.

而这一切绝对不单是我个人的功劳。虽然是由我发起的，但现在我们拥有一个大约 4,000 名开发者的社区。它已经拥有了自己的生命力。他们包揽了从 CI/CD 到方方面面的所有事情。这太不可思议了，其规模远超我个人。我真的得加快节奏了，我准备的内容太多了。

英文原文

And

it's

it's

definitely

not

me.

Like

I

I

started

this

and

now

we've

got

a

community

of

like

4,000

developers.

This

has

taken

on

a

life

of

its

own.

They

are

doing

everything

from

CI/CD.

It's

it's

incredible.

This

this

is

way

bigger

than

just

me.

Uh

I

really

got

to

pick

up

the

pace

here.

I've

made

way

too

much

content.

社区正在开发超过 300 款应用。我们开发了一款应用，让非技术人员也能开始参与贡献，并且可以基于我们构建的应用底座（比如 UI/UX 库）来进行创作。我们正在把这些解构为所有的组件和图元，以便大家复用。这样大家就能拥有出色的 3D 库、出色的 2D 库，这一切都非常棒。

英文原文

So

um

there

over

300

apps

being

developed

by

the

community.

We've

developed

an

app

where

non-technical

people

can

start

contributing

and

use

the

basis

of

the

apps

that

we've

created

like

the

UIUX

libraries.

We're

comp

decomposing

this

into

all

of

the

the

components

and

primitives

so

that

you

can

reuse

them.

So

you'll

have

nice

uh

3D

libraries,

nice

2D

libraries.

It's

all

great.

技术栈与开发模式

接下来我谈谈技术层面的内容。Rust 对于这个项目的成功大概起到了核心关键作用。我们以前见过有人用 Electron 来做，但对于高品质的创意应用，你确实需要原生的东西。Rust 具备大量极其优秀的品质。谷歌曾发表过关于其缺陷率极低的论文。它的启动速度非常快，是做这类项目最好的语言。

英文原文

I'll

get

into

the

technical

stuff.

So

um

Rust

was

probably

essential

in

this

product

project

success.

We'd

seen

people

do

it

in

Electron

before,

but

for

high

quality

creative

apps,

you

really

need

something

that's

native.

Um,

there's

just

a

lot

of

really,

really

good

quality

things

about

Rust.

Google

had

papers

about

like

the

low

defect

rate.

It

starts

really

fast.

It's

it's

the

best

language

for

this.

对，它支持所有不同的平台和操作系统。我们把它移植到了 NetBSD 上，移植到了平板电脑上，它还能通过 WASM 原生部署到 Web 端。因此它无处不在、无所不至。

作为一名工程师，今天想要开始使用 Rust，是否必须精通 Rust 呢？绝对不需要。我本人从 1.0 之前的时代就一直在用 Rust，但团队里的每个人……其实是一群从未接触过这门语言的人混合在一起。你的智能体能非常顺畅地驾驭它，你不会遇到任何麻烦。它是一门非常出色的语言。你唯一会遇到的大问题就是编译时间。

“现在能做 GUI 了吗？”（is it GUI yet?）在 Rust 领域有这么一个著名的网站（或一系列网站）。它是否支持现代特性？是的，你完全可以构建富 GUI 应用和 Rust UI/UX。

英文原文

Uh,

yeah.

Um,

it

supports

all

all

different

platforms

and

operating

systems.

We've

got

this

on

NetBSD.

We've

got

this

on

tablets.

It

natively

deploys

to

the

web

via

WASOM.

So

it

can

get

anywhere

and

everywhere.

Um,

do

you

need

to

know

Rust

to

start

using

Rust

today

as

an

engineer?

Uh,

definitely

not.

Uh,

I've

been

using

Rust

since

pre10

times,

but

everybody

on

the

team,

uh,

it's

a

mix

of

people

that

have

never

experienced

the

language

before.

Your

agents

will

drive

it

just

fine.

You're

not

going

to

have

any

trouble.

It

is

a

fantastic

language.

The

one

big

thing

you're

going

to

run

into

is

compile

times.

Um,

is

it

gooey

yet?

There's

this

famous

website

or

series

of

websites

in

the

Rust

Rust

world.

Does

it

support

like

modern

features?

Yes,

you

can

build

guey

rich

gooey

apps

and

and

Rust

UIUX.

我们踩过很多坑，比如权限问题、GitHub、CI 的扩容问题。很多人都讨论过这些。我得快速过一下这部分，因为时间快不够了。CI 简直快崩了，我们一直在自己搭建环境。我们找到了一个性价比很高的地方来获取托管服务器上的大内存，目前正在迁移过去。

这个点很有意思：智能体开发 vs 人工评审。如果你在做后端工作，或者涉及模式迁移（schema migrations）、API 设计之类的事情，让你们的人类工程师来做。但如果是桌面应用、游戏，以及任何本质上不可变的内容，直接把智能体扔上去，给它们充分的信任。你完全不需要全程让“人类在环”（human-in-the-loop）——比如上游去全程指挥方向，直接放手让它跑，指示它去构建东西就行。我们社区里有人正在用 40 台机器用 Rust 重写虚幻引擎（Unreal Engine）。所以你根本不用去操心这些事。

英文原文

Uh

we've

run

into

a

lot

of

gotchas

like

permissions,

GitHub,

scaling

CI.

A

lot

of

people

have

talked

about

that.

I'm

going

to

just

speed

through

this

because

I'm

kind

of

running

out

of

time.

Uh

CI

is

like

falling

over.

We've

been

spinning

up

our

own.

We

found

a

good

place

to

get

RAM

on

colloccated

servers.

We're

moving

to

that.

Um

this

is

a

good

one.

So

like

agentic

develop

versus

human

uh

review.

If

you're

doing

back-end

work

or

things

that

require

schema

migrations,

API

design,

let

your

humans

do

that.

But

for

like

desktop

apps

and

games

and

anything

that's

basically

immutable,

throw

your

agents

at

it

and

give

them

like

full

confidence.

You

don't

need

to

be

human

in

the

loop

fully.

Like

upstream

like

directing

where

things

go,

just

just

let

it

run.

Tell

it

to

build

things.

Somebody

in

our

community

is

is

building

uh

they've

got

40

machines

rebuilding

Unreal

Engine

in

Rust.

So

you

you

don't

even

have

to

mind

these

things.

AutoCraft 是一个完全自主的软件工厂，我们即将把它开源。它能导入 issue，对它们进行分流排期，执行智能体开发，对所有决策进行后续校验评判。这个很快就会开源。大体上我们拥有了一个完全用代码设计的应用工厂，你可以去描述和表达它。我们很快会将其开源。我时间超了，只能加快点速度。接收 issue……

[掌声]

这是开源的。Code Rabbit，一切皆开源。我们做了很多工作。[笑声] 我原本以为自己会留出更多时间来讲技术细节，但这些内容非常棒，请大家务必关注。它将支持你们所有的智能体开发，支持你们去规划和组织如何开展一个项目的全套方式，让你们的工程师保持专注在任务上。再次强调，完全开源。所有的……

英文原文

Um,

AutoCraft

is

a

completely

autonomous

software

factory

that

we're

going

to

open

source.

It

ingests

issues.

It

triages

them.

It

does

agentic

development.

It

postrains

any

valves

on

all

of

the

decisions.

This

will

be

open

source

soon.

Um,

basically

we've

got

an

entire

app

factory

that's

designed

in

code.

You

can

express

this.

We'll

open

source

this

soon.

I'm

just

going

to

have

to

hurry

because

I'm

over

time.

Uh,

injust

issues.

[applause]

This

is

open

source.

Code

rabbit

open

source

all

the

things.

We're

we're

doing

a

lot

here.

And

[laughter]

I

I

thought

I

would

save

more

time

for

the

technical,

but

this

is

really

good

and

please

please

look

out

for

this.

Um

it

will

support

all

of

your

agenting

development.

It

supports

all

of

the

sort

of

crafting

and

curating

how

you

uh

approach

a

project.

It

keeps

your

engineers

on

task.

Um

again,

fully

open

source.

All

of

the

由它做出的决策，就是我们将用于决策树评估和后训练的训练数据。嗯，让我们夺回整个互联网。让我们重建一切：所有软件，所有平台。

英文原文

decisions made by it are training data that we're going to use for evals and post-training on the decision trees. Um let's take back the entire internet. Let's rebuild everything. all software, all platforms.

[applause]

英文原文

[applause]

[applause]

英文原文

[applause]

我们可以——我们可以把所有这些东西变成规范的协议。嗯，我们可以开源整个世界。让我们拥有自己的应用程序、基础设施、协议、模型，所有的一切。嗯，我们正在招人。所以如果有人感兴趣，想做 100% 的开源工作——我们的代码没有一行是闭源的，全部都在公开环境中。嗯，呃，你会有时间做自己的项目，只要通过 hello@storyteller.ai 联系我就行。非常感谢大家。

英文原文

We can we can turn all of these things into proper protocols. Um we can open source the entire world. Let's own our apps, our infra, our protocols, our models, all of it. Um we are hiring. So if anybody's interested and wants to work on 100% open source, none of our code is closed source. All of it is in the open. um uh you get time to work on your own projects, just reach out to me at hello@storyteller.ai. And thank you so much everybody.

[applause]

英文原文

[applause]

上午主题演讲收尾与串场

我们的上午主题演讲已接近尾声，是时候请回我们的主持人来进行收尾了。女士们，先生们，请再次欢迎 The Duckbill Group 的首席云经济学家 Corey Quinn。

英文原文

We're nearing the end of our morning keynotes and it's time to bring back our MC to wrap things up. Ladies and gentlemen, please welcome back Corey Quinn, chief cloud economist at the Duck Bill Group.

刚才那场演讲可真难接。我不太确定自己该怎么接，而这意味着我根本不必去接。靴子。主持最好的一点就在这儿。第一步：简短。呃，这是一个不可思议的上午。呃，我喜欢这些早间主题演讲的地方在于，它们往往能把现场气氛调动起来。它们帮我们开启新的一天，并为我们带来——呃，工作、想法和对话。现在我们走出去查看新闻，却发现刚才谈论的内容有四分之三已经变得无关紧要了，因为从那时起又发布了四个新模型，成立了两家新实验室。但它引领我们进入各个专题会议，接下来我们有一些很棒的分会。呃，稍后我们还会回到这里会合。呃，不过当你在外面的时候，多跟人聊聊。我——我最喜欢的会议环节之一就是走廊交流。跟人们聊聊他们正在做什么，聊聊你正在做什么。呃，尽量别向别人过度推销。有些人在这方面确实需要提个醒。嗯，与老朋友重新联系，并在我们进入这最后一天之际，在整个社区中保持这份活力。所以，如果我在外头没碰到你，是因为你在和其他真有实用内容可讲的人交谈，那太棒了。呃，我们将在今天下午的闭幕主题演讲环节在此相聚。呃，今天下午 4:20 开始。谢谢大家。去吧，玩得开心，做出些有趣的东西来。

英文原文

Now, that was a hard act to follow. I'm not sure how I'm supposed to do that, which means I don't have to. boots. That's the best thing about MCing. Step one, be brief. Uh it was an incredible morning. Uh the thing that I love about these morning keynotes is that they tend to kick the energy up a bit. They help us start the day and they give us with uh work and ideas and conversations. And now we go back out there and we check the news and find out that threequarters of what was just talked about has now irrelevant because there's been four new model releases and two new labs founded since then. But it carries us into the sessions and we have some great ones coming up. Uh we're going to meet back here afterwards. Uh but while you're out there, talk to people. My my one of my favorite parts of conferences is the hallway track. Talk to people about what they're working on. Talk about what you're working on. U try not to overly pitch people too hard. Some people need reminders on that. Um reconnect with old friends and and keep the energy going as well across the community as we get into the final day here. So, if I don't see you out there because you're talking to other people with actually useful things to say, great. Uh, we'll see you back here for our closing keynote block. Uh, kicks off at 4:20 this afternoon. Thank you. Go forth, have fun, and build something interesting.

做得好。

英文原文

>> Good job.

谢谢你。非常感谢。

英文原文

>> Thank you. Appreciate it.

Heat. Heat. N.

英文原文

Heat. Heat. N.

好。

英文原文

All right.

Heat. Heat.

英文原文

Heat. Heat.

Heat. Heat. N.

英文原文

Heat. Heat. N.

嘿，嘿，嘿。

英文原文

Hey, hey, hey.

Heat. Heat.

英文原文

Heat. Heat.

为 Agent 设计 CLI 工具

嗨，我是 Aayush。我构建了一个 CLI，可以让你的 agent 在不同工具之间复用以往会话的上下文。在改进它的过程中，我意识到了为什么我的 agent 暗地里讨厌我的 CLI，以及你可以对你的 CLI 做些什么。

英文原文

Hi, I am Aayush. I built a CLA that lets your agent reuse context from your past session across tools. While improving it, I realized why my agent secretly hated my CLI and what you can do about yours.

Claude Code 想从一个文档页面获取一项配置。于是它下载了整个页面，抓取了它，并把结果保留在 40 行。

英文原文

Cloud Code wanted one setting from a docs page. So, it downloaded the whole page, grabbed it, and kept the result at 40 lines.

Codex 想知道有多少依赖项存在漏洞。它请求了整个 JSON 报告，并保留了最后 40 行，也就是该报告放置总量数据的地方。

英文原文

Codex wanted to know how many dependencies were vulnerable. It asked the whole JSON report and kept the last 40 lines where this report puts the totals.

Cursor 运行了一个测试套件，发现了一处失败，并保留了最后 80 行。这三次截取都保留了此处显示的信息。每一次截取都内嵌了一个假设，即答案位于输出的哪个位置，而这可能会对构建这些工具的人隐藏摩擦。单次管道命令能告诉你的很少。一次又一次写出的同一个管道命令，则可以指向你的工具所缺失的东西。这些都是来自我自己转录文本的真实调用，而 Claude Code 那次是 agent 在评审这场演讲，它依然忍不住这么做了。

英文原文

Cursor ran a test suit, found one failure and kept the last 80 lines. All three cuts kept the information shown here. Each one encoded in an assumption about where the answer sits in the output that can hide friction from the people who build these tools. One pipe tells you very little. The same pipe written again and again can point to something your tool is missing. These are real calls from my own transcripts and the cloud code one was the agent reviewing this talk and it still couldn't help itself.

Agent 可以在不提交 bug 报告的情况下绕过摩擦。它们围绕你的程序编写一个小程序。6 月，内容平台 Sanity 撰文介绍了他们的 MCP 服务器。自发布以来，有超过 20,000 个 agent、约 300 万次工具调用，且没有收到一份 bug 报告。于是他们推出了一个提供反馈的工具。一旦 agent 有了抱怨的地方，它们就开始表达哪里出了问题。但你不需要反馈工具才能开始。反馈已经存放在你转录文本的某个地方了。

英文原文

Agents can work around friction without filing a bug report. They write a little program around your program. In June, Sanity, the content platform, wrote up their MCP server. More than 20,000 agents, about 3 million tool calls since launch and not a single bug report. So they shipped a give feedback tool. Once agent had somewhere to complain, they started expressing what's wrong. But you don't need a feedback tool to start. Feedback is already sitting somewhere in your transcripts.

我——我从自己相同的转录文本中提取了大约 20,000 次 agent 调用。那个样本中出现了 14 个 CLI：Git、GitHub CLI、cargo、curl、npm、docker 等等。四分之一将输出导入了诸如 head、grep、tail、python、sort 或 wc 等过滤器中。对于读密集型命令，这一比例超过了一半。git show 和 cargo test 各占 62%。有些测试框架管道用得少得多。例如 Codex 允许 agent 在每次调用时设置输出上限。

英文原文

I I extracted about 20,000 agent calls from my same transcripts. 14 CLI appear in that sample. Git, GitHub CLI, cargo, curl, npm, docker, and others. A quarter fed output into filters such as head, grab, tail, python, sort or wc. For read heavy commands, it's more than half. Git show and cargo test 62% each. Some harnesses pipe far less. Codeex, for example, lets the agent set an output limit on each call.

而且管道不仅会在批处理中削减噪声而管道未见失败。管道会返回其最后一个命令的退出代码。因此假设你把 cargo test 用管道传入 head。即使测试失败，它也可能报告成功。屏幕上就是一个例子：编译错误加上一个绿点。状态是错误的，但 agent 仍然能读取该错误。

英文原文

And pipes don't only cut noise in batch without pipe failed. A pipeline returns the exit code of its last command. So let's say you pipe cargo test into head. It can report success even when the test failed. Here's one example on the screen. A compile error and a green dot. The status was wrong but the agent could still read the error.

管道还可能把错误本身截断，那就更糟了。在 GitHub 上的一个 Claude Code issue 中，一位用户报告说某个脚本抛出了引用错误并带有错误代码退出。tail 显示了三行无害的内容，Claude 报告了成功。该错误从未传达到模型。所以我给出的建议是在工具失败时大声报错。把最后几行做成简短的摘要，以及指向完整日志的路径。

英文原文

A pipe can also cut the error itself which is worse. In a cloud code issue on GitHub, a user reported that a script through a reference error and exited with an error code. tail showed three harmless lines and Cloud reported success. The error never reached the model. So my advice would be to fail loudly when a tool fails. Make the very last lines a short summary and a part to the full log.

所有这些都是我在琢磨自己构建的一个 CLI 时学到的。这是一个小工具，能让编码 agent 跨 Claude Code、Codex、Cursor 搜索它们自己过去的会话。它附带一个 skill，以便 agent 能够决定何时以及如何调用它。Agent 就是我正在研究的用户。我 10 月份的审查记录了大约一千次调用，这就是我的小型案例研究。这种方法适用于你的 agent 所调用的任何东西。只需看看它们在调用之后紧接着做了什么。

英文原文

I learned all this while tinkering with a CLI I built. It's a small tool that lets coding agents search their own past sessions across cloud code, codeex cursor. It ships with a skill so the agent can decide when and how to call it. Agents are the users that I'm studying. My October audit recorded about thousand calls and that's my small case study. The method works on anything your agents calls. Just look at what they do right after the call.

80%。Agent 用我的工具打开旧会话，十有八九输出直接进入了 grep、sed、tail 或 Python。我称之为管道税。管道可以避免无用信息进入上下文，这没问题。税在于 agent 每次都要亲自动手编写那个一模一样的削减命令。每次的消耗都很小，而且大体上行得通，但每一次都是对答案在哪里的又一次猜测。

英文原文

80%. Four out of five times an agent opened an old session with my tool. The output went straight into grab sad tail or Python. I call that the pipe text. A pipe keeps junk out of context and that's fine. The text is that the agent writing that same cut down command by hand every single time. It's small each time and it mostly works but each one is another guess about where the answer is.

一个合理的反对意见来自我一直在读的 Hacker News 上关于为 agent 设计 CLI 的帖子。该评论称应该由 agent 自己选择要读取多少输出。Agent 完全可以顺畅地通过 grep 或 head 使用管道过滤。嗯，它们确实可以，而且对于一次性操作它们也应当如此。我并不反对这一点。但在那些日常 CLI 中，我发现 83% 的管道都符合至少出现过 10 次的模式。相同的 CLI、子命令和管道工具。传入的参数可能不同，但 agent 总是在索取相同的东西。

英文原文

The fair objection comes from a hacker news thread about designing CLS for agent that I was reading. The command says to leave it to the agents to choose how much output to reads. agent can pipe through crapper or head perfectly well. Well, they can and for oneoff they should. I'm not against that. But across those everyday CLI, I found 83% of pipes fell into the patterns seen at least 10 times. The same CLI subcommand and pipe tools. The arguments passed could differ, but the agent keep asking for the same things.

我知道它们所截减的输出可能很大。我用我的工具以两种方式打开过一个长会话。整个内容有 120 万个字符，这就是为什么 agent 要用管道——仅搜索匹配项周围的部分大概是 3,000 个字符。测试框架确实会对输出设限，但大小限制并不能识别出相关的消息。另一方面，选择标志可以让 agent 告诉你的工具它想要哪一部分。

英文原文

I understand the output they're cutting is probably big. I opened one long session with my tool two ways. The whole thing 1.2 million characters and that's why the agent pipe just the part around the search match was about 3,000 characters. Harnesses do cap output but a size cap does not identify the relevant messages. Selection flags on the other hand let the agent tell your tool which part it wants.

因此根据我的经验以及对我自己工具所做的改进，我来给你们举三个例子。请把左边一列看作是 bug 报告。从第一个开始，使用 rg 和 sed 过滤意味着“让我挑选我想要的部分”。所以看我的命令。因此我的读取命令 view 现在可以仅返回匹配项周围的部分、仅返回匹配的消息，或者带有大小限制的最后几条。

英文原文

So based on my experience and improvements I made to my own tool, I'll give you three examples. Read the left column as the bug reports. So starting with the first one, filtering with the RGN set means let me pick the part I want. So read my command. So my read command view can now return just the part around a match only the matching messages or the last few with a size gap.

第二个例子是在 JSON 之上使用 Python。在 JSON 之上使用 Python 意味着“给我更少的字段”。因此搜索增加了一种紧凑模式，每条命中一行，在得出相同结果的情况下比 JSON 小大约六倍。

英文原文

Second example would be Python over JSON. So Python over JSON means give me fewer fields. So search got a compact mode one line per hit about six times smaller than JSON with the same results.

聊天历史记录中的 inspect 命令用于概括一个会话，大约三分之一的 inspect 调用来自 agent 手动遍历多个会话。正如你看到的带有 for 循环，那意味着“让我一次查询多个”，现在这变成了一次调用。如你所见，我的路线图就蕴含在围绕我命令的那些命令之中，而我必须从中学习。

英文原文

Inspect command in chat history summarizes a session about the third of inspect calls came from agent looping over several sessions by hand. As you can see with the for loop that means let me ask several about several at once and now it's one call. As you can see my road map was in the commands around my command and I had to learn from that.

云成本工具 Infracost 在 5 月份写下了相同的发现。他们的 agent 经常在 JSON 转储之上编写 jq 和 python 管道。因此他们把过滤移入了 CLI，并添加了一种紧凑格式以及一个用于指导它的 skill。在他们的测试中，没有任何这些优化的原生 Claude 在六道难题中答对零道，而有了它之后答对了六道中的六道，并且整个基准测试的成本也下降了 41%。在 Hacker News 上，写那篇帖子的人说阅读那些 agent 日志就像在看可用性测试，我完全同意。

英文原文

Infraost the cloud cost tool wrote up the same discovery in May. Their agents were writing jq and python pipelines over JSON dumps. So they moved the filtering into the CLI and added a compact format and a skill to teach it. Plain Clawude with none of that got zero of six hard questions right in their testing and with it six of six and cost across the benchmark also fell 41%. On Hacker News, the person who wrote that post said that reading those agent logs felt like watching usability tests and I couldn't agree more.

这一部分可能

英文原文

This part might

听起来微不足道且显而易见。一旦它进入 CLI，就可以通过 skill 文件或 help 标志教你的 agent 它的存在，但 agent 会从字面上理解这些教学。我做了一些更改，把我的 deep 标志变成了空操作（no-op）。我保留了告诉 agent 使用它的 skill，想看看它们会怎么做。它们会发现还是不会？95 次搜索中有 95 次仍然传递了该标志。在那之后，两个 session 加载了重写的 skill，不再告诉它们使用它。没有任何搜索传递该标志，并且两者在第一次读取时都尝试了新标志。因此，你的 skill.md、你的 help 文本、你的错误信息就是接口。这也意味着过时的文档就是 bug，因为你的 agent 会一直遵从它们。

英文原文

sound trivial and obvious. Once it's in the CLI, teach your agent it's there with the skill file or the health flag, but agents take the teaching literally. I made some changes that turned my deep flag into a no op. I left the skill telling the agents to use it to see what they do. Would they find out or not? 95 searches of out of 95 still passed it. After that, two sessions loaded a rewritten skill and that no longer told them to use it. None of the searches passed the flag and both tried new flags on their first read. So your skill.md, your help text, your error messages are the interface. That also means stale dogs are bugs because your agent will keep following them.

一旦新标志传递给 agent，几乎每次搜索都使用了紧凑输出，并且几乎每次都减少了新标志。管道读取（pipe read）从 82% 下降到大约四分之一。

英文原文

Once the new flags reached agents, nearly every search used compact output and nearly every reduced the new flags. Pipe breed fell from 82% to about a quarter.

你可以轻松为你的工具获取真实的关卡和测试用例。如果你每天都使用它，它们就已经存在于你的 transcript 中了。agent 实际运行的命令就是测试用例，而 agent 紧接着执行的操作就是一个粗略的标准答案。在你的旧构建和新构建中重放这些命令。如果你更改了排序，检查 agent 接着使用的那个结果呃是否仍然排在最前面。如果你更改了输出格式，检查是否有内容丢失。如果你更改了默认值，查看哪些命令现在的表现有所不同。而当 agent 读取器使用不同的参数进行调用时，你可能就发现了一个失败用例。标准答案只是对 agent 所需内容的粗略猜测。保留几个用例，并用它们来给你的修复打分。

英文原文

You can get real levels and test cases for your tool easily. If you use it every day, they are already in your transcripts. The commands agent really ran are the test cases and what agent did immediately next is a rough answer key. replay those commands through your old build and your new one. If you change a ranking, check that result uh the agent went on to use still comes out on the top or not. If you change an output format, check that nothing went missing. If you change a default, see which commands now behave differently. And when an agent reader has a call with different arguments, you have probably found a failing case. The answer key is only a rough guess at what the agent wanted. keep a few cases back and use those to score your fix.

所以，在分析了 20,000 次调用之后，如果让我重新开始，这就是我会首先构建的内容。测量的管道中有 55% 在下游仅使用了 head、tail 或 sed，这表明了对更小切片的需求。在做任何花哨的事情之前，先展示范围和限制。一个标志只有在 agent 伸手去用它时才有用。GitHub CLI 的 list 命令有一个 limit 标志。agent 在 81% 的时间里使用了它。反例是 cargo，对于 cargo test，quiet 模式下每个测试打印一个字符而不是一行，并且仍然报告失败。agent 大约在每 10 次 cargo 运行中有 1 次传递了它，其余时候则使用了管道。因此，要在你的 help 文本和你的 skill 中教这个标志，并让大输出默认打印得更少，并说明如何获取更多。例如，以类似于“showing 20 of 300, add offset flag 20 for the next page”的内容结尾。

英文原文

So after analyzing 20,000 invocation, if I were to start all over again, this is what I would build first. 55% of the measured pipelines used only head, tail, or set downstream that points to demand for smaller slices. Show ranges and limits before anything fancy. A flag only helps if agents reach for it. GitHub CLI's list command have a limit flag. An agent used it 81% of the time. The counter example would be cargo for cargo test quite mode prints a character per test instead of a line and still reports failures. Agent passed it on about one cargo run in 10 and pipped instead. So teach the flag. So teach the flag in your help test and your skill and make big outputs print less by default and say how to get more. for example, and with something like showing 20 of 300, add offset flag 20 for the next page.

一次性交付其中几项的一种方法是 agent 模式。你的 CLI 会注意到有 agent 正在调用它，并以不同的方式打印。像 Claude Code 这样的 agent 会在其 shell 中设置环境变量，以便你的工具能够识别它们。DuckDB 发布了一个。在终端中，你会得到一个绘制精美的表格，但 agent 会得到一个纯文本表格，上面说明省略了多少行以及如何获取它们。超大查询会提前停止并予以说明，错误则作为 JSON 返回，因为 agent 能更好地读取它们。这与我们一直以来的列表相同，确实如此，它将模型读取的输出减少了 59%。

英文原文

One way to ship several of these at once is an agent mode. Your CLI notices an agent is calling it and prints differently. Agents like cloud code set an environment variable in their cells so your tool can spot them. Doug DB shipped one. At a terminal, you get a nicely drawn table, but an agent gets a plain one that says how many rows were left out and how to get them. A huge query stops early and says so and errors come back as JSON because agents read them better. Is the same list as we have been just true and it cut the output model red by 59%.

然而，节省的 token 几乎只占模型读取的所有 token 的 0.5% 左右。所以成本和运行时间并没有改善。如果你还记得基础设施成本，agent 开始答对问题。DuckDB 本就已经是这样了。因此，要衡量轮次和正确答案，而不仅仅是 token。

英文原文

However, the tokens saved were nearly about.5% of all the tokens the model read. So cost and runtime didn't improve. If you remember infra cost agent started getting answers right. Doug Dvs were already there. So measure turns and write answers not just tokens.

你可能会想，我会让我自己的 agent 来做所有这些事情。它们在这方面还不是那么擅长。至少无法单次生成（oneshotting）搞定。设计正是出错的地方。让 agent 为 agent 构建工具，它可能想象的是一个人，然后把 agent 硬套上去。它会复制它所见过的最常见模式，而其中大部分模式都很糟糕。一项针对官方 MCP 服务器的研究发现，大多数只是包装了现有的 API。另一项检查来自真实 MCP 服务器的工具描述，并在其中 97% 的描述中标记了问题。文档也存在同样的差距。苏黎世联邦理工学院的研究人员测试了由 agent 编写的上下文文件。它们并没有帮助任务成功，运行成本更高，并且表现比人类编写的文件更差。agent 已经擅长本次演讲的另一半内容了。给它一个 transcript 和一个它可以运行的测试，它就能找出问题所在并改进工具。有时甚至超过了人工编写的水平，Anthropic 在其自身内部工具上也报告了这一点。因此，由你来决定工具应该是什么样，让 agent 根据实际发生的情况去对照检查。

英文原文

You might be thinking I'll ask my agent to do all of this. They are not that good at this yet. At least not oneshotting it. The design is where it goes wrong. Ask an agent to build a tool for agent. It probably pictures a person and then bolts the agent on. It copies most common patterns it has seen and most of those are poor. A study of official MCP servers found most just rep an existing API. Another check tool descriptions from real MCP servers and flagged issues in 97% of them. Docs have the same gap. Researchers at each Zurich tested context file written by agents. They didn't help tasks succeed cost more to run and did worse than files written by people. Agents are already good at other half of this talk. Give one transcript and a test it can run and it can find what's wrong and improve the tool. Sometimes past what people wrote by hand and tropic reported that on its own internal tools as well. So you decide what the tool should be and let the agent check it against what really happened.

你们已经看到了每一项的证据，所以我就不再逐一念了。花点时间吸收这些内容，或者截个图。为了找到你自己的列表，首先备份你的 transcript，因为 Claude Code 可能会在 30 天后删除不活跃的记录。然后搜索工具名称后面跟着管道以及其他类似模式的内容。如果你发布了一个 MCP 服务器，留意 agent 立即用不同参数再次调用你的工具的情况。如果你从这次演讲中记住一件事，那就是如果你正在为你的 agent 构建工具，你的 agent transcript 就是你可能没有去阅读的产品分析数据。

英文原文

You have seen the evidence for each of these so I won't read them. Take a moment to absorb these or screenshot. To find your own list, back up your transcript first because clot code can delete inactive once after 30 days. Then search for them for a tool's name followed by a pipe and other similar patterns. If you ship an MCP server, look for agent calling your tool again straight away with different arguments. If you remember one thing from this talk, if you are building a tool for your agent, your agent transcript are the product analytics you probably aren't reading.

变通方法就是 bug 报告。所以去阅读你的 transcript 吧，如果你在做这件事时需要任何帮助，可以在 Cargo 或 GitHub 上查看 chat history。它也可以为你做这件事。谢谢大家。

英文原文

The workaround is the bug report. So go read your transcript and if you need any help doing that, check out chat history on cargo or GitHub. It can do this for you as well. Thank you.

会话数据收集与自愈基础设施

大家好，感谢大家来听我的演讲。我是 B Douggee，Brian Douglas，我将要讲的是如果你打算在口头上辱骂你的 agent，至少给结果打上标签。现在，这是一个我将快速过一遍单一例子的演讲，而且这将是一个简短的演讲。嗯，主要是因为这里面其实没有太多的东西，但是嗯，除此之外，我我想谈谈我在过去呃六到八个月里一直在做的一些工作。嗯，这实际上是在今年早些时候产生的想法。谈到 agent，大家肯定都像是被喂了“AI 药丸”。嗯，显然 Claude Code 肯定是目前最受欢迎的工具之一。我们还有相当多的其他可以利用的工具，比如 Codex 或 ChatGPT。嗯，多年前我读过一本书，嗯其实是相当多年前了，呃是 Rick Rubin 写的关于《The Creative Act: A Way of Being》。这实际上是围绕着这样一个理念：Rick Rubin 制作了 Beastie Boys 以及许多其他演出组合的作品。呃，但他其实从未演奏过音乐。呃，很多人将这与 vibe coding 联系起来，Rick Rubin 可以在不实际弹奏钢琴的情况下制作出最好的音乐。呃，我们现在有了一个我们身处的世界，人们可以在从未写过 Rust 的情况下 vibe code 出一堆 Rust 代码。呃，而这也正是我目前的状态。呃几个月前还有另一篇文章，呃来自 Addy Osmani，他现在目前在呃嗯 Anthropic 团队，具体是在 Claude Code 团队，呃里面谈到了初级工程师的机会，或者说基本上这机会已经不复存在了。嗯说到初级工程，从初级成长到高级完全在于反复练习（reps）。因此，你获得的练习次数越多，你获得的参与度越高，基本上就越有机会提升作为一名工程师的水平。呃，而没有了这些练习，我们正把这些练习专门交给了 agent。呃，我们正在失去提升水平的机会。嗯，所以这源于我今年早些时候做的一件事，嗯，构建了这个名为 tapes 的开源项目。Tapes.dev 是基于 Apache 2 开源的。嗯，市面上还有很多其他类似的东西，但就像我想要表达的，重点不在于 tapes。这是关于收集你的 agent 会话数据的理念。这就是它的作用。所以，呃，很多 Otel 收集器，很多会话捕获服务。呃，很多 harness 已经开箱即用地为你做到了这一点。只是想提一下这是我使用的工具。你可以使用你喜欢的任何其他工具。嗯，除了 tapes 之外，我我和我的团队，我们开发了这个叫做 stereos 的东西。这是你的 agent 的运行时。再说一次，沙箱并不是今天每个人都需要听到的概念，但只要知道在我一直在做的所有工作中，我一直在捕获这些会话，并且呃通过拥有一个可复现的环境来记录它们。呃这个环境本身，它们有这些信号，并且呃通过 tapes 和 stereos 的这个小设置，我一直在做的一件事就是构建自愈基础设施。所以，呃这是一个名为 Sweeper Agent 的小项目，而 Sweeper Agent 本身对我来说是一种构建并行 agent 的方式。嗯目前我可以将这些扩展到多达大约 10 个。我已经做了呃相当多的

英文原文

>> Hey folks, thanks for coming to my talk. I'm B Douggee, Brian Douglas, and I'm going to talk about if you are going to verbally abuse your agents, at least label the outcome. Now, this is a talk I'm going to quickly just run through one example, and this will be a short talk. Um, mainly because there's actually not too much too much to this, but um, outside of that, I I want to talk about some of the work I've been doing for the past uh, six to eight months. Um, and it actually came out with this idea early this year. Everyone definitely got AI pill when it comes to agents. Um, so obviously cloud codes got to be one of the most popular ones out there. We've got quite a few other ones that you can leverage like the codeex or chatbt. Um, and there was a book I read years ago, well quite a handful of years ago, uh, from Rick Rubin about the creative act, a way of being. And this was actually around the idea that Rick Rubin produced the BC boys and a number of other acts. Uh, but he's actually never played music. Uh, and a lot of folks have made this correlation with vibe coding where Rick Rubin can produce the best music without actually playing the piano. Uh, we're now having a we're in a world where folks can vibe code a bunch of Rust code without never writing Rust. Uh, and that's exactly where I'm at. Uh there was another article um months ago uh from Adi Osmani who's now currently at uh um the anthropic team on the quad code team specifically uh and it was talking about the the opportunity for juniors or basically this it's gone. Um when it comes to junior engineering it was all about growing from junior to senior was about reps. So the more reps you got the more engagement you got opportunity to basically just level up as an engineer. Uh, and without those reps, we're handing those reps specifically to agents. Uh, we're losing the opportunity to level up. Um, so this comes out of something I was doing early this year, which um, built this open source project called tapes. Tapes.dev is open source Apache 2. Um, there's a lot of other similar stuff out there, but like what I'm getting at is it's not about tapes. This is about the idea of collecting your agent session data. That's what this does. So, uh, a lot of hotel collectors, a lot of session capture services. Uh, a lot of the harnesses already do this for you out of the box. Just wanted to mention that this is what I use. You can use anything else you would like. Um, I've also in addition to tapes, we me and my team, we've developed this thing called stereos. And this is the runtime for your agent. Again, sandboxes not a concept that everyone needs to hear about today, but just knowing that in all the work I've been doing, I've been capturing these sessions and uh recording them by having a repro reproducible environment. Uh and this environment themselves, they have these signals and uh the one thing I've been doing it with these this little setup of tapes and stereos is I've been building self-healing infrastructure. So, uh this is a little project called Sweeper Agent and Sweeper Agent itself is a way for me to build parallel agents. Um I can scale up to 10 about 10 of these at this point. I've done uh quite a few

扩展代理任务与 Rust 编码挑战

额，比如 Daytona 沙盒也行。我曾经同时运行大约 200 个沙盒，专门去解决某个特定的问题。所以像写文档、编写测试、代码清理这类你想扩展规模的工作，是非常明确的任务，你希望同时派发给多个代理协同处理。现在，这其中的很大一部分工作，都是在清理我过去大概 18 个月里通过“氛围编码”（vibe coding）写出来的软件。不过，我只是想指出来，我们其实有很多工作可以做，来从根本上解决这些问题。

英文原文

uh like Daytona sandboxes as well. I've done like about 200 at the same time that will just solve a specific problem. So documentation, test writing, code cleanup, um stuff that you want to scale that's like very very defined work that you want to spin off to multiple agents uh in tandem. Now a lot of this has been cleaning up most of my vibe coding um software that I built in the last uh what 18 months at this point. Um, but just pointing out that like there's a lot of work we can do to basically solve these problems.

额，但这改变不了我依然不知道怎么写 Rust 代码的事实。我自己从来没有专门写过 Rust 代码，或者说没有专业地写过。额，10 年前我确实写过 Rust 代码，但那主要是为了学习 Rust 做的一些小副业项目。但在我工作的公司 Paper Compute，我们日常都在使用 Rust。因为底层有这项叫做 tapes 的技术，我们一直在捕获跟踪数据（traces）。tapes 就是我们的跟踪捕获和遥测捕获服务，而在 Paper，我们有一个控制台。所以现在我先切出幻灯片，给大家展示一下 Paper 控制台。

英文原文

Uh, but it doesn't change the fact that I still don't know how to write Rust code. I've never written Rust code uh specifically myself. Uh, professionally rather. Uh, 10 years ago, I did write Rust code, but it was mostly for little side projects to learn Rust. Um, but we use Rust day-to-day at the company I work for, which is paper compute. Uh and what we've been doing is because of this technology under underneath this called tapes uh we've been capturing traces and tapes is our trace capture service telemetry capture uh and paper we have a console. So for the moment I'll move away from the slides and show you the paper console.

从审查与反驳（Push Back）中提取知识

Paper 控制台本身只展示了我能够捕获会话。在本地运行与这样做不同的地方在于，我可以拥有一个集中式的数据存储，不仅是我，我的队友也能用。额，而且很酷的一点是我还可以给会话打标签。那么，这是一个……哦，有意思。不好意思，得缩小一下视图才能看到标签。不过是的，在这里我可以看到一个会话，它正在处理某些 Paper 云导出内存限制的问题。我们有这样一个词，一个叫做“push back”（反驳）的标签。

英文原文

Now paper console itself just showing that I can capture sessions. The difference of doing this local in my own sessions is that I can have a centralized data store like not just me but also my teammate. Uh, and what's cool about this is I can also label sessions. So, uh, this is a session that, um, oh, interesting. Yeah, sorry. Got to zoom out to see the labels. But yeah, so here I can see I've got a session here which uh working as some paper cloud export memory limits. And we've got this word, this label called push back.

现在，我专门进入了一个带有“push back”标签的会话。额，这具体记录了每一次工程师反驳代理的情况。最近有这么一件事，我想是 Anthropic 那边，或者说是传言吧，我不知道这到底有没有事实依据，但传闻说 Anthropic 正在抵制……如果你对代理不礼貌，你可能会失去账号或被封禁。但我特别发现，在反驳这方面，当我专门去查看 Matt 写的 Rust 代码或 Go 代码时，每当他提出反驳，这就表明他掌握着代理所不知道的知识。

英文原文

Now, I actually went into specifically a session that had push back in it. Uh, and it's specifically every time an engineer pushes back on the agent. Now, there's there's a thing that uh I think recently Anthropic or it was I guess rumors. I don't know if this is actually grounded by any any facts, but uh Anthropic was pushing back on uh if you're being rude to the agent, you could potentially lose your account or get blocked. Um but what I find specifically when push back um when I'm specifically looking for any Rust code or Go code that Matt has written uh and whenever he pushes back, it's a good signal that he has knowledge that the agent does not.

这种情况通常表现为：尽管 Rust 编译器能编译通过，但我们决定以某种特定方式构建 API。我们已经决定了代码应该如何组织，以及该代码应该如何与代码库的不同部分进行交互。此外，反驳还包括指出诸如重复实现某些服务的情况，而代理可能并不了解相关的上下文。还有很多其他原因，但我真正想表达的是，作为所谓的“初级”Rust 工程师，我没有足够的背景知识来向代理提出反驳，去对它说：“嘿，你这里做错了，或者你做的选择很奇怪。”我通常会对代理用 Rust 写的任何东西全盘照收。

英文原文

And this usually comes in the form of we have decided our our API is structured certain way despite the fact that the Rust code compiler compiles. We have made decisions on how that code should be set up and how that code should interact with different parts of the codebase. Also pushing back on things like duplicating uh services that maybe the agent didn't have context around. There's a lot of other like a bunch of reasons, but what I'm getting at, me as like quoteunquote a junior Rust engineer, uh I don't have enough context to be able to push back to the agent to say, "Hey, you're doing this wrong or you're making like weird choices." I usually take everything that the agent writes in Rust for face value.

这就导致了当我发起 PR 时，在代码审查阶段会受到人类工程师的大量反驳。这主要是因为大家知道我不写 Rust 代码。对于代码库，甚至在审查的时候，我能够提供的反驳并不多。所以当代码送到 Gretile 那里时，即使是五星好评的审查意见，我也就直接放行了。那么，我该如何避免这种反驳呢？我所做的实际上是开始收集每一个这样的标签和跟踪数据。然后我要指出的另一点是，控制台本身捕获了每一个会话，我拥有所有的具体细节。这实际上只是它的命令行版本。我一直在做的就是捕获所有这些会话，然后放入一个打好标签的数据集中。

英文原文

And what that ends up turning into is when I open up PR, I get a lot of push back from the engineers, the human engineers at review time. Uh this is mainly because we know I don't write Rust code. There's not a lot of push back I can provide to the codebase. um or even at the time of review. So by the time it gets to Gretile, it's a five out of five review. I just let it slide. Um so how do I prevent that push back? What I've done is actually started collecting every single one of these labels and the traces. Uh and then one thing I'll point out is that the the console itself, I capture every single session. I got all the uh specific details. This is actually just a command line version of that. Uh what I've been doing is I've been capturing all these sessions and then dropping into a data set that's been labeled.

构建 Twin Reviewer 与 Harness

这样一来，我现在就可以直接运行一个名为 twin review 的小型命令行工具。我会指定一个 PR 编号，比如 381，然后运行它，它就会真正去遍历并从本质上审查代码，就像是 Matt 在审查一样。我现在实现这一点的方式就是捕获跟踪数据。我创建了数据集，并且也做了一些后训练（post training）。这里你可以自由选择实现方式，但我专门针对 Matt 在 Go 和 Rust 代码中提出的每一次反驳做了一些后训练。我让它在我的 GitHub PR 上以审查者 Matt 的身份为我提供评论。

英文原文

Uh so now I can just like run a little command line tool called twin review. Um and I'll grab a a PR number 381 and I'll run that and it'll actually run through and essentially review the code as if it was Matt. Now the way I'm doing this is I'm capturing the traces. I created the data set. I've also done some post training. Uh and this is like where you know choose your own adventure but I've done some post training uh specifically on every time Matt pushes back within Go Rus code. Uh and I I asked for it to provide me comments as a viewer Matt on my GitHub PR.

所以，这些数据完全只存在于跟踪记录中，并没有采用任何 GitHub PR 的审查数据。我认为大家可能也可以采用那种模式。但我发现的是，我们团队的大多数人已经开始在命令行上审查代码了。你会让 Claude 去审查代码，或者让 Pi harness 去审查代码，或者你去审查这个 PR，但你不会在 GitHub 本身上面进行大量的反驳或撰写大量评论。虽然我们现在所看到的是，审查代码的方式正在转变——以前代码专门由人类在 GitHub 上审查，现在则是由代理审查；而现在，代码变成了完全在你的 harness 中进行审查，没有任何监督、评论，或者任何直接导向你们共享代码库的内容。

英文原文

So uh this is all data that's only specifically within the traces. So this is not taking any GitHub PR review data. Uh which I think you potentially could also follow that pattern. Uh but what I find is like the majority of our team have started reviewing code on the command line. So you'll have Claude review the code or the Pi harness review code um or you review this PR but you don't do a lot of push back or a lot of commentary on GitHub itself. Though what we're seeing right now, we're shifting from code being reviewed specifically on GitHub by agents. Well, previously humans and now agents to now code is specifically this getting reviewed in your harness without any sort of oversight or commentary or anything that sort of navigates directly to your um your shared codebase.

话虽如此，我摸索出来的是，我创建了这个叫做 twin reviewer 的东西，而且这个也是开源的。我刚才没打开这个界面，不过……噢，实际上 twin reviewer 并没有开源。我意识到我从来没有将它开源，因为里面包含敏感数据。但我所做的，实际上是使用 Fireworks 搭建了同样的模式，专门利用 Fireworks 来做后训练。很酷的是，我寻找了各种反驳、道歉、各类优质会话（即那些确实被标记为你可能应该注意的会话），以及针对各种通常会导致代理崩溃的情况的回归测试。

英文原文

Um so with that being said, like what I've navigated is I've created this thing called the twin reviewer and this this is well is is open sourced. Um, so it is when I did not have this open, but um, oh actually twin reviewer is not open source. I realized I never open source it because it's got sensitive data. Um, but what I've done is actually set up that same pattern using fireworks and using fireworks to post train uh, specifically. Uh, and what's cool about this is like I've I've looked for any sort of push backs, apologies, uh, any sort of golden sessions, sessions that actually um have been marked uh, as you should probably pay attention to this and also regressions for anything that uh, usually agents will just fall over.

很多时候，情况会变成代理开始重复造一些已经存在的 Go 语言库。遇到这种情况通常肯定是要反驳的。理想情况下你应该反驳它，但话说回来，如果你不知道这些库的存在，你大概就会直接照单全收，然后就会重复造出类似 Kubernetes 之类的东西。这听起来可能有点夸大其词，但我们确实见过它重复造诸如用于测试的 mock 库等非常流行、知名代码库或非常流行的开源项目中的东西。

英文原文

A lot of times it turns into um, like an agent will start reinventing Go libraries that already exist. Uh, that usually is granted to be pushed back. Like I ideally you should push back on that but again if you don't know these libraries exist you probably just take it for face value and then you're reinventing things like Kubernetes uh which sounds like an overstatement but like we've seen it reinvent things like mock libraries for for testing and etc. uh and very popular codebase or very popular open source projects.

从 Skills 与 Evals 入手

基本上就是这样了。简单来说，如果你打算在代码审查中提出反驳，并且拥有非常有价值的见解，我的建议是去寻找跟踪数据中的机会，或许可以借此训练或后训练一个模型，或者在本地为你自己的代码审查搭建一个小型的 harness。你可以自由选择路径。我认为你完全可以通过自己构建 harness 来完成其中的大部分工作。不过如果你确实想使用一个 Qwen 38B 模型（在这个场景下我具体就是这么做的），你也可以采用这种方法。

英文原文

So that's essentially it. This is uh basically if you're going to push back and you're going to have some really really valuable insight uh under reviews in your codebase. My recommendation is take a look at opportunities within the traces uh to maybe train or post-train a model or create your own little harness for reviews locally. Choose your own adventure. I think you could actually do a lot of this by just building your own harness. Um but if you did wanted to use a um a Quinn 38B, which I've done uh specifically uh in this situation, you'd also approach that.

你可以使用很多现成的东西和工具。我最初做的事情，其实是开始提取 Paper 的跟踪数据来创建 skills。这些 skills 能够具体执行诸如检查 tapes 之类的操作。如果你有一个数据库——具体到 tape 来说，它是我机器本地的一个 Postgres 数据库，所以你并不一定要使用 Paper 控制台。实际上 Paper 控制台目前还在排队等待名单中。因此，最好的方案可能是使用像 tapes、Braintrust 或类似工具来捕获你的跟踪数据。

英文原文

Uh a lot of the stuff is available to you, a lot of tools. Uh what I did actually to start is I started taking the paper traces and creating skills um skills that I can then specifically think do things like check the tapes. So if you have a database which tape specifically is a postgress database local on my machine. So you don't have to use the paper console. Actually paper console is currently weight listed. So probably best use case is going to use uh something like tapes or a a brain trust or something similar to capture your traces.

有了 skills 之后，很酷的一点就是你可以创建 eval（评估）。一旦有了 eval，就可以在 Qwen 模型上进行后训练，用于特定目的，比如识别 Rust 代码是否符合整个组织编写代码的水平。我确实写过一篇关于训练模型而不是编写 skill 的博文。我的建议可能是先编写一个 skill。如今你很可能可以从一个 skill 中获得最大的效用。我们整个团队拥有大约将近 80,000 个会话，我们可以从中创建 skills，这非常棒。这件事情很酷的地方在于……

英文原文

Um and then what's cool about this is once you have skills you can create eval. Uh once you have eval post training on a Quinn model uh and for specific reasons for like identifying if R code is up to up to speed to how you writing code across your organization. Um so I did write a blog post around training a model instead of writing a skill. My recommendation is probably write a skill first. You can probably get most mileage uh out of a skill uh today. Um and I specifically across my team we had about almost 80,000 sessions um that we could create skills from which is pretty cool. Cool thing about this

专用微调与直接偏好优化

还有就是做专用微调（specialized fine-tuning）或者直接偏好优化（direct preference optimization）。后者会需要多得多的算力，或者更大的、某些大型硬件来完成。所以大概会选专用微调。呃，如果你了解或者说这就是我做的方式，我是在像 5090 这样较小的 GPU 上对其进行量化。呃，如果你不了解专用微调，这就像是在书的页边写批注。如果你的书就是你的模型，那么页边的批注就是你的权重，呃，或者更确切地说是你的适配器（adapters）。也就是说，在已经预训练好的模型之上生成的那些权重。很好。正如大家所见，我是在一个 Quinton 4B 模型上完成这一步的。呃，效果挺不错的。呃，我确实试过在一些更大的模型上做这个。呃，特别是偏好优化，耗费了大量的算力。

英文原文

is also you do specialized fine tuning or direct preference optimization. The latter is going to take a lot more compute or larger some large hardware to do that. So probably specialized fine tuning. Uh if you know about quick or if that's how I'm doing this I'm quantizing it on smaller GPUs like a 5090. Uh, and if you don't know about specialized fine-tuning, this is like writing notes in the margin of a book. If your book is your model, the notes in the margin are your weights, uh, or your adapters rather. So, the weights you generate on top of the the the model that's already been pre-trained. Cool. And I did this as as you saw on a Quinton 4B model. Uh, it works pretty good. Uh, I did try to do this on some larger models. Uh, so specifically for preference optimization, it took a lot of compute.

我唯一能搞定这个的时候，就是在 Brev 服务上使用英伟达 GPU，或者其实 fireworks 上的某些东西我一直都没拿到访问权限。所以我只用了较小的 6000 系列。嗯，但如果你想知道，直接偏好优化——如果说页边批注是你的专用微调，那么 DPO 就会像是你的精华摘要（cliff notes）。呃，在小型语言模型或小型 4B 模型上是不可能做的。呃，你会需要 70B 以上的规格，呃，这得花上一些硬件。呃，就我个人而言，我本地没有任何这种硬件，所以我不得不使用嗯我提到的那些服务，Fireworks，还有——好吧，甚至都算不上 Fireworks，Brev 是我当时唯一能获得访问权限的。如果你能拿到更大硬件的访问权限，祝你好运，呃，好吧，你得为这些东西预付很多费用。所以，在这点上祝好运了。

英文原文

The only time I could make this work is by using Nvidia GPUs on the Brev uh service or actually I never got access to the some of the stuff on fireworks. So I was only did smaller 6000s. Um but if you if you care to know direct preference optimization if if the margin notes are your specialized fine-tuning DPO is going to be like your cliff notes. Uh not not possible doing a small language model or a small 4B model. uh you would need like 70B plus uh which is going to take some hardware. Uh me specifically, I didn't have any as hardware local, so I had to use um the services that I mentioned, Fireworks and well, not even Fireworks, Brev was the only one I could get access to at the time. Good luck if you can get access to larger uh well, you got to prepay a lot for the stuff. So, best of luck on that one.

嗯，不过我的建议是构建 skills。呃，如果有兴趣可以去试试 tapes.dev，或者利用你手头现成的工具。呃，在此提醒一下，呃，你或许应该读读这本书。如果你思考一下你知道我们是如何展望未来、在涉及 AI 和氛围写代码（vibe coding）以及利用智能体基本构建起你的软件工厂时我们正如何应对这个世界的，这是一本极好的书。呃，而且我确实建议请去看一下 AIM 的博客文章。呃，他在 X 上有这么一个系列，呃，我觉得都很棒。呃，一定要去看看。他有一些思路清晰的想法，呃，值得一读。呃，我是 B Douggee，非常感谢大家观看这个视频。我尽量讲得简短，因为内容相当直截了当。希望确实如此。再会。

英文原文

Um but my suggestion is create skills. Uh try out tapes.dev if you're interested or leverage the tool that you have at your ready. uh in reminder uh you should probably read this book. This is a great book if you think about you know how we're looking forward into how we're approaching this world when it comes to AI and vibe coding and leveraging agents to basically build your software factory. Uh and I do recommend please check out AIM's uh blog post. Uh he has a series that these on X uh I think they're all great. Uh definitely take a look at them. He's got some some clear clear-minded thoughts uh to check out. Uh I'm B Douggee and I do appreciate you watching this video. I try to keep it brief because it was pretty straightforward. Hopefully it was. Cheers.

VLM Serving 配置优化智能体

好的。大家好。我的名字是 Rohan，这是关于最近构建出来的优化智能体的一个快速演示。先快速介绍一下这个智能体的目的是什么、目标是什么，然后我将进入简短的演示，以及我们刚才运行的一次前期运行的结果。我们这里的目标是构建一个智能体，帮助针对任意给定的模型找到最佳的 VLM 服务配置。呃，目的是减少手动寻找最佳配置和 VLM 服务方案（recipe）所花费的时间。如果我们能将其自动化，让它在后台运行，并且让智能体更快地找到方案和优化参数，那么它将节省大量时间。这就是该智能体的目标，接下来详细说明一下该智能体实际上是如何做到这一点的。

英文原文

Right. Hello everyone. So my name is Rohan and this is a quick demo on the optimization agent that what has been recently built. So a quick intro into what the purpose is and what the goal is of this agent and then I'll get into a brief demo and results that uh from a previous run that we just did. So what the goal is here is to build an agent that helps um find the best VLM serving config for any given model. Uh the purpose is to reduce the time it takes to manually um find the best configs and the VLM serving recipe. And if we can automate this and have it run in the background and um have the agent find the recipes and the optimized parameters faster, then it will save a lot of time. So that's the goal of the agent and a little bit more into how the agent actually does this.

智能体所采用的策略叫做贝叶斯优化（Bayesian optimization），贝叶斯优化是一种用于在任何给定情况下寻找最理想数值的技术。所以在我们的场景下，就是为了在 VL1 方案中找到最理想的数值。呃，它采用数学步骤，从根本上缩小了搜索空间。因此，通过缩小搜索空间，它大大减少了在运行期间寻找该方案中优化数值所花费的时间。贝叶斯优化是所使用的策略，而 Optuna 是在搜索循环中运用该策略的工具。Optuna 使用贝叶斯优化。它向其输入当前的方案，即初始方案。呃，贝叶斯优化在此基础上运行，经过多次迭代或多次试验，它最终找到了优化后的方案，而 Optuna 基本上在每次试验后获取每个方案，对其进行评估，获得一个分数，运用策略来修改方案、改进它，然后将其重新放回搜索循环中运行，直到试验完成。

英文原文

So the strategy that the agent uses is called Beijing optimization and Beijian optimization is a technique um that's used to find the most optimal numbers um in any given situation. So in our situation it's to find the optimal numbers in the uh VL1 recipe. uh it uses mathematical steps that essentially minimizes the search space. So by minimizing the search space, it uh reduces a lot of the time that it takes in finding the optimized uh numbers in this uh recipe during the run. So B uh basian optimization is a strategy that is used but optuna is the tool that uses a strategy in the search loop. So, Optuna uses Beijian optimization. It feeds it the current recipe, the starting recipe. Uh, Beijian optimization runs on that and over many iterations or many trials, it eventually finds the optimized recipe and Optina basically takes each recipe after every trial, assesses it, gets a score, um, uses a strategy to change the recipe, improve it, and then it reruns it within the, uh, search loop um, until the trials is done.

那么我们来稍微深入看一下我们是如何操作的。我现在正在这里的一个 droplet 实例中，例如这个 droplet 加载了 GPT OSS 12B 模型。我们可以做的就是查看当前的方案。我们在这里看到的是我们用来测试它的四种工况（regimes）。有 chat short，有 chat long，有 rag 和 batch classify。呃，它们有各自输入的一组 ISL 和 OSL token。呃，所有这些都可以根据我们想要看到的结果来进行更改。但本质上，这就是你在下载模型权重时会获得的初始配置、初始方案。

英文原文

So let's go a little bit into how uh we do this. So I'm currently in a droplet here and this droplet for example has GPT OSS uh 12B model loaded up. So what we can do is uh we can look at the current um recipe. So what we see here is four regimes right that we're testing it on. So there's chat short, there's chat long, there's rag and batch classify. uh they have their own set of ISL OSL tokens that we're feeding it through. Uh all these um can be changed depending on what uh we want to see. But essentially this is the starting uh config starting recipe that you would get when you download the weights of the model.

嗯，接下来一旦你的 droplet 下载好了模型，并且你准备好运行这个智能体，你就可以粘贴进这条命令，呃，它基本上会告诉你权重的来源在哪里。嗯，激活一个虚拟环境并运行名为 tuner.py 的脚本。呃，这个脚本基本上需要你指定几项内容。你需要给它指定研究名称（study name）。你需要告诉它初始的 YAML 文件，也就是我们刚才在上面看到的那个。你需要告诉它运行所在的硬件。你还需要告诉它你想要的试验次数（trials）。呃，在这个示例中我们设定了 24 次。嗯，我简短地展示另一个过去运行过的示例，那次只运行了 9 次试验。所以，嗯，无论如何，这就是运行的命令。呃，通常这些命令需要运行几个小时。我想对于 GPL，上面显示运行了大约四到五个小时，然后对于另一个模型，也花了几个小时。

英文原文

Um so moving once you have the model downloaded in your droplet and you're ready to run this agent you paste in this command uh which essentially tells you you know where the where the weights are coming from. um activate a virtual um environment and have um uh or not have but run the script called tuner.py. Uh this script essentially tells you or it has a couple of things you have to tell it. So you you want to give it the name of the study. You want to tell it the starting uh YAML file which we just saw above here. You want to tell it the hardware that it's running on. And you want to tell it the number of trials that you want. So uh in this example we have 24. um briefly I'll show you another example of a past run that was run uh that had only nine trials. So um uh anyway uh this is the command that's that's run. Uh typically these commands take um a couple hours to run. I think for the GPL it says it took about four or five hours and then for the other model it took a couple hours as well.

所以我们可以查看智能体完成工作后生成的 HTML 报告。嗯，这一份是针对我刚才提到的 GPOSS 模型的。呃，我们在 HTML 报告中看到的是总共多少次试验中，哪一次试验是最佳试验。在当前的案例中，是进行了 50 次试验。呃，所以在运行结束时你会得到一个优化后的 YAML 文件，其中包含了我们目前在 BLM 服务方案中寻找的所有数值。在这个案例中，我们发现——或者说智能体发现 256 是优化后的数值，呃，依此类推，而它获取准确度的方式是衡量生成的有效吞吐（goodput）token 数量。所以总的来说，我们发现相比最初的配置，在生成的 token 方面有 17.5% 的提升，而且正如我提到的，这一份是针对 GPOSS 的。我们这里还有另一份 HTML 报告，是针对 deepseek R1 distill 模型的。在这里我们运行了 9 次试验，在这 9 次试验中，第 9 次是最好的。呃，我们发现 token 吞吐量翻倍了，而且 ITL 改善了将近一半，也就是降低了将近一半。这就是关于你运行的命令以及输出整体长什么样的快速介绍和简要演示，你会得到一份告诉你每次试验进展情况的 HTML 报告，然后最终你还会得到一份包含优化方案的 YAML 文件。希望这些内容有所帮助。谢谢。

英文原文

So we can go into the HTML report that's generated after the uh agent finishes its work. Um so this one is for the GPOSS model that I mentioned. Uh what what we see in the HTML report is which trial number was the best trial out of the total of how many trials. So in this case it was 50 trials. uh you get so at the end of the run you get an optimized uh YAML file which includes all the numbers that we currently look for in the BLM serving recipe. So in this case we found or the agent found that 256 was the optimized number uh so on and so forth and the way that it's getting the accuracy is the amount of goodput tokens that's being generated. So uh overall we found that from the first config we had an improvement of 17.5% in terms of uh tokens generated and this one as I mentioned is for GPOSS. We have another HTML report here which is for the deepseek R1 distill uh model. In here we found that or in here we had uh nine trials that was run and out of the nine the ninth one was the best. uh we found that token throughput uh doubled and ITL was improved by almost half aka it was reduced by almost half. So this is the quick intro and brief demo on the commands you run and what the output looks like in in in general you get a HTML report that tells you um how each trial went and then you also get um a YAML file that has the optimized recipe at the end of the day. So hopefully that was helpful. Thank you.

管理大型 AI 编程项目的熵增

嗨，我是 Dimmitri。在本次分享中，我将展示我是如何开展大型 AI 编程项目的维护工作的。编码智能体让写代码变得廉价，也让熵增变得更加廉价。在漫威电影中，灭霸眼睁睁看着自己的母星泰坦星在自身的无序膨胀下走向崩溃。因此，当宇宙变得拥挤时，他抹除了所有生命的一半。这次演讲讲的是我在一个项目中做出同样举措的那一次经历。我们的代码当时就像泰坦星那样不断膨胀。当我发现智能体干了什么之后，我删掉了没人用得上的测试和样板代码。我不得不重写整个项目，并且丢失了一些有用的组件。这就是两难之境：要么淹没在混乱中，要么破坏掉人们正在使用的功能。在这里，我们将为那个项目构建第三种选择。重写并没有维持太久。智能体一直在写，混乱又重新滋生。所以我在其上方划了一条界线。变动之多，已经超出了任何人的审查能力。第三种选择是保持在其承载能力之下，进行小范围清理

英文原文

Hi, I'm Dimmitri. In this session, I will show how I work on maintaining large AI coding projects. Coding agents made code cheap. They made entropy even cheaper. In the Marvel movie, Thanos watched his home planet Titan collapse under its own growth. So, when the universe got crowded, he raised half of everything. This talk is about the one time I did the same in one project. Our code was growing the way Titan did. When I found what my agents had done, I deleted tests and boil plate code nobody used. I had to rewrite the whole project and I lost some useful pieces. And that's the dilemma. Drown in chaos or break what people use. And here we'll build the third option for that project. The rewrite didn't hold. The agents kept writing and the mass grew back. So I drew a line above it. There is more change than anyone can check. The third option is to stay under it with small sweeps

每周以及内置于系统中的反馈回路。这次演讲的每一章，都是我为了走到那一步必须学会的一件事。隧道原本需要六颗宝石才能前进一步，而我们的则是六个问题，它们会随着我们的推进在角落亮起。第一个问题是时间：我们如何保持在红线之下？

英文原文

every week and feedback loops built into the system. Each chapter of this talk is one thing I had to learn to get there. Tunnels needed six stones for one step. Ours are six questions and they light up in the corner as we go. The first one is time. How do we stay under the line?

第一章：正确性由什么决定

第一章。为了保持在红线之下，我首先必须弄清楚在这里“正确”意味着什么。我过去以为这由测试来决定。那么正确到底取决于什么？

英文原文

Chapter one. To stay under the line. I first had to know what right means here. I used to think that tests decide that. So what does right actually depend on?

让我们在脑海中记住一个任务：有人想要修改一笔我们已经批准的订单上的金额。这里会遇到两类智能体：一类负责编写指令，另一类则在生产环境中执行它们。

英文原文

Let's keep one task in mind. Someone wants to change the amount on an order we already approved. Two kinds of agents meet here. One wrote the instructions and the other follows them in production.

我们在 110 个国家运行这个系统，涵盖了约 10,000 个纯文本自然语言指令场景。每个任务都会落入一个分片中：一个国家中的一个流程。这里有第二颗宝石——空间。它询问修改被允许扩散到哪里。默认情况下，它会停留在自己的分片内部。

英文原文

We run this in 110 countries with about 10,000 scenarios of pros instructions. Every task lands in a slice. One process in one country. There's a second stone space. It asks where a change is allowed to go. By default, it stays inside its slice.

该系统的大部分是纯文本语言描述，据我估计大约占四分之三。所以文本表达同样需要工程化。刚开始我们借鉴了变异测试：你故意改坏一句话，然后观察测试是否变红。在这里，几乎没有任何测试变红。如果你的包装层全是代码，那么你的智能体指令和测试就是你的自然语言文本。对于智能体来说，重写很容易，难点在于知道重写后必须保留什么。对于代码，测试可以告诉你；而对于文本，只有人能告诉你。智能体无法独立构建复杂的系统。它们只修复容易看到的问题，并在此过程中破坏一致性。而人也会疲劳，所以我们只在必要的地方消耗他们的判断力。当业务方与团队交流时，背后隐藏着一个真正的问题：这是一个需要人工处理的步骤，还是我们可以将其自动化？业务专家无法回答这个问题，他们看不到系统内部是如何连接的。因此，开发人员利用他们的代码编写智能体深入排查追踪链和日志，由他们来决定属于哪一种。在我们的任务中，团队主管的决策仍然保留人工处理。其限制在于我们能够关联的数据。

英文原文

Most of this system is pros about three quarters by my estimate. So pros needs engineering too. At start we borrowed mutation testing. You break one sentence on purpose and you watch for red test. Here almost none turned red. And if your wrapper is all code, your agent instructions and your tests are your pros. With agents, a rewrite is easy. The hard part is knowing what has to survive it. For code, tests can tell you. For pros, only people can. Agents can't build a complex system on their own. They fix what is easy to see and break consistency on the way. And people also get tired. So we spend their judgment only where it's needed. When business ask with the team, it hides a real question. Is this a human step or can we automate it? Experts can't answer that. They don't see how the systems connect inside. So developers dig through the traces and logs with their coding agents and they decide which it is. In our task, the team leads decision stays human. The limit was data we could connect.

业务专家看到的是规则，开发人员看到的是水面之下的内容，而大部分缺陷正是潜伏在水面之下。他们让代码编写智能体把那些关联写下来，以便运行时系统能够进行检查。我们给专家的指南表述得非常简单：业务方告诉我们“做什么”，我们来搞定“怎么做”。一份文档，面向两类读者。智能体阅读代码块，所以这些区块必须精确；专家阅读周围的文本，并在上面签字确认。我们只保留一份副本，第二份副本会产生偏差，并且这一机制在两端双向运作。我们的代码同样需要向专家提供通俗的描述。专家只能修正他们看得见的东西。他们每个人审查任务的方式与智能体看到它的方式完全一致。他们知道这会成为代码编写智能体做出的下一次修改，并在同一个任务上进行测试。因此，正确性取决于人。没有任何单个人能够全知全能。由一组专家来做决策，这就是为什么我们从一开始就把他们引入进来。你会在这里经常看到灵魂宝石。我们会在第四章讨论它。

英文原文

Business experts see the rules. Developers see what's under the water. And that's where most bugs leave. They have their coding agents write those links down so the runtime can check them. Our guide for experts says it simply business tell us what we work out how. One document to readers. Agents read the blogs. So the blocks have to be exact. Experts read the text around them and they sign it. We keep one copy, the second one with drift and works both ways. Our code needs a plain description for experts too. Experts can only fix what they can see. Each of them reviews a task exactly the way the agent saw it. They know it becomes the next change made by a coding agent and test it on the same task. So right depends on people. No single person knows it. A group of experts decides and that's why we bring them in from the start. You'll see the soul stone here a lot. We take it in chapter 4.

第二章：在何处仍需人类判断

第二章。专家一次修正一个任务，而智能体则整天运行同样的文本逻辑。我们追踪每一个被接受或拒绝的结果。准确率向我们展示了哪些地方仍然需要人工介入。我过去以为目标是将一切都搬到代码中。那么，我们在哪里仍然需要判断力？再次回到我们的任务：文本说明写着上限是 1,500，而代码写的却是一千。同一个订单在一个地方没问题，在另一个地方却超限了，但没有任何东西抛出错误。甚至像“不要调用 X”这样的一行字，也会让模型知道 X 的存在。这就是第三颗宝石——现实。它是真的，还是对模型来说仅仅看起来是真的？自然语言文本没有编译器，所以我们自己构建了一个。然后，作者将每个修改后的文件进行静态检查，就像代码一样记录进账本。

英文原文

Chapter two. Experts correct one task at a time while agents run the same pros all day. We track every outcome accepted or rejected. The accuracy shows us where people still need to look. I used to think the goal was to move everything into code. So where do we still need judgment? Here is our task again. The pro says the limit is 1,500. The code says a th00and. The same order is fine in one place and too high in the other and nothing throws an error. Even a line like do not call X teaches the model that X exists. That's the third stone reality. Is it true or does it only look true for the model? Pros has no compiler. So we built one. Then the author lints every changed file into ledger just like code.

静态检查无法阻止多份副本之间发生漂移，所以我们观察执行轨迹。一条轨迹就是一个智能体穿行于文本逻辑中的路径。当智能体不断走在同一条路径上时，我们就会将其转化为一个工作流工具，并删除走过该路径的文本描述。我们最初的工作流工具把事情搞得更糟了：文本里仍然列出了工具内部的步骤，导致智能体把那些步骤又重复执行了一遍。一旦文本中只保留决策，它就奏效了。现在，一条执行轨迹只有在它不断重复且在各个国家之间保持一致时，才会变成代码，否则它就依然保留为文本形式。那些仍然存在差异的部分，作为文本保留在框架之内。代码负责准备事实数据并执行决策。专家拥有决策表的所有权，而模型只负责挑选某一行。如果没有合适的行，就由人工接管。下一步我们还没有构建完成：模型将承担微小的决策，并由工作流将它们串联起来。这个边界是双向移动的。我们将数百行表达式从代码移入文本逻辑中，因为专家看不懂代码；而正则表达式用来识别关键信号依然很好用。当数量变多时，它们就会被移到配置中。在专家能够读懂的前提下，只要答案存在变数，判断力就保留在那里。

英文原文

Lind can't stop copies drifting aparts. So we watched the walks. A walk is the path one agent takes through the pros. When agents keep walking in the same path, we turn it into one workflow tool and we delete the pros that walked it. Our first workflow of tools made things worse. The pros still named the steps inside them. So agents run those steps again. It worked once. The pros kept only the decisions. Now a walk becomes code only when it repeats and agrees across countries otherwise it stays pros. What still varies stays in pros inside the frame. Code prepares the facts and carries out the decision. Experts own the decision table and the model only picks a row. If no row fits, a person takes over. We haven't built the next step yet. The model will take the small decisions and workflows will connect them. This boundary moves both ways. We moved hundreds of lines of expressions out of code and into pros because experts couldn't read them. A reax is still fine for spotting a critical signal. When there are many, they move into config. Judgment stays wherever the answer varies in a place an expert can read.

第三章：保持受控的真正含义

第三章。现在由模型做出决定，其决定会转化为行动。我过去以为控制意味着对智能体的权限控制。那么，保持受控到底意味着什么？

英文原文

Chapter three. Now the model decides and its decision becomes an action. I used to think control meant permissions on the agent. So what does it mean to stay in control?

控制始于模型能看到什么，这就是第四颗宝石——心灵。模型知道什么？当我们让它自主选择加载什么时，它把所有东西都加载进来了。然后仅仅因为价格信息可见，它就凭空捏造了一个价格校验。任何留在其上下文中的内容都会变成行动的理由。对于我们的任务，由运行时环境决定模型能看到什么：那是来自订单的三个事实数据。完整记录保留在运行时环境中，模型犯错的举动就会减少。以下是我们如何减少大型集合的上下文膨胀：模型传递一个句柄，即运行时环境已经持有的数据的一个简短名称。在发生任何写操作之前，运行时环境会对其进行校验。这既保持了上下文体量小巧，又足以支持做出决策。与此同时，它还会针对一致性对上下文进行校验。一个掌握正确事实的模型仍然可能造成破坏。这就是第五颗宝石——力量。什么引发了修改？我们在开发期间以及在运行时写入时对修改进行双重控制。只有代码能打开写入路径，因此我们每天可以修改文本逻辑很多次。那些开放的环，就是仍然依赖于人的部分。让我们放大来看看执行写入：五道防护墙立在它前面，第六道则检查它是否已落地。如果某次写入看起来失败了，智能体就会重试，日志流水确保它只执行一次。控制的意义远不止于对智能体的权限设置。

英文原文

Control starts with what the model sees. That's what the fourth stone mind. What does the model know? When we let it choose what to load, it loaded everything. Then it invented a pricing check just because prices were visible. Whatever sits in its context becomes a reason to act. The runtime decides what's the model sees for our task. That's three facts from the order. The full record stays with the runtime and the model makes fewer wrong moves. Here how we minimize context blo for a big set. The model passes a handle, a short name for date that the runtime already holds. The runtime checks it before any write. It keeps the context small and still enough for the decision. At the same time, it uh validates context for consistency. A model with the right facts can still do damage. That's the fifth stone, power. What made change? We control a change twice. during development and again at the right at runtime. Only code opens the right path so we can change pros many times a day. The open rings are the parts that still depend on people. Let's zoom in and run right. Five walls stand in front of it. And the sixth checks that it landed. If a right looks like it failed, an agent retries and the journal makes sure it runs only once. Control means much more than permissions on the agent.

第四章：注意力保护与系统失控的信号

第四章。大多数日子里我都掠过系统表面。从高处看，一切显得井井有条，因此我不会深入到智能体所做的每一处修改中。接着，表面开始变得粗糙。我们如何知道自己正在失去控制？

英文原文

Chapter four. Most days I glide over the system. From up here, everything looks fine. So I don't go down into every change my agents made. Then the surface gets rough. How do we know we're losing control?

所有那些控制机制保护了系统，却没有保护我的注意力。一开始，我阅读每一处修改，系统也正是借此吸收了我的习惯。后来，变更的体量增长到了单个人无法全部阅读的程度。我陷入了精疲力竭，不再阅读每一处修改。那就是最后一颗宝石的代价。那么，由谁来评判，代价又是什么？有了它，无限手套才算完整。我认识的每一位技术主管都经历过这个境地：成为所有变更都在等待的那一个人。智能体只是让你在几周内而不是几年内就陷入这个局面。我需要一种不依赖单个人头脑的工作方式。而第一个衡量指标就是我自己：当我感到疲惫且无法解释修改了什么时，那就是信号。在该指标告警的同时，仪表盘可能全都是绿色的。所以我停下来，先修复流程，而不是等待某个数据指标出现。我无法再阅读每一份 diff，因此我构建了用于监控事物变化规律及其导数的仪表工具。其中之一就是爆炸半径。在我们构建一个新的检查项之前，我们曾在数千个历史会话中回放过其规则，它原本触发的概率还不到 1%。在数值发生跳变的地方，我会深入下去仔细阅读。当某些东西看起来有问题时，我们会等待：单个信号会等到它再次出现，即便它来自一位专家也是如此。而在多个分片中出现的集群问题则不同，那时我们会统一修复一次其背后的概念。如果听任不管，智能体会不断添加条件，直到今天的任务全部通过；当你把这些条件移除时评测得分会下降，而你将得到一份真实的不足清单。

英文原文

All those controls protect the system. Nothing protected my attention. At the start, I read every change and that's how the system picked up my habits. Then the volume grew past what one person can read. I was burning out and I stopped reading every change. That was the price of the last stone. So, who judges and what does it cost? With it, the gauntlet is complete. Every tech lead I know has been here. the one person every change waits on. Agents just get you there in weeks instead of years. I needed a way of working that doesn't run through one head. And the first gauge was me. When I'm tired and I can't explain what changed, that's the signal. The dashboard can be green while that gauge is firing. So, I stop and fix the process first. I don't wait for a number. I couldn't read every div anymore. So I built instruments that watch how things change and their derivatives. One of them is blast radius. Before we built one new check, we replayed its rule over several thousand past sessions. It would have fired on less than 1%. Where the numbers jump, I go down and read. When something looks wrong, we wait. One signal waits until it comes back. even when it comes from an expert. A cluster across slices is different. Then we fix the concept behind it once. Left alone, an agent keeps adding conditions until today's tasks pass. The eval score drops when you take them out and you get an honest list of gaps.

一个标红的分片很少意味着……专家的批注会变成一项评测，然后由智能体在同类分片中搜索相同的失败案例。同类分片因市场和当地法律的不同而有意存在差异，因此我们对它们进行排查，且修复只下发到已确认存在失败的地方。

英文原文

One red slice is rarely a low. The expert's note becomes an eval and an agent searches the siblings for the same failure. Siblings differ on purpose by market and by local law. So we sweep them and the fix goes only where the failure is confirmed.

在任何时刻都有 10 到 20 个这样的回路保持开启。在某一刻，我已经变成了用户与……之间的消息传递者。

英文原文

At any time 10 to 20 of these loops are open. At one point I had become the message pass between users and mine.

Heat.

英文原文

Heat.

Heat. N.

英文原文

Heat. N.

Heat. Heat.

英文原文

Heat. Heat.

好的。大家好。呃，今天很高兴能来到这里。呃，先做个简短介绍：我是 Walden，我是联合创始人之一兼……

英文原文

All right. Hello everyone. Um, it's a pleasure to be here today. Uh, quick introduction. I'm Walden. I'm one of the co-founders and

内部研发吞吐量与软件工厂实践

Cognition 的首席产品官。我们打造了一款名为 Devon 的 AI 产品。不过今天我大部分时间想从产品本身抽离出来一点，重点讲讲我们在工程团队内部是如何运作以大幅提高速度的。作为这一切的背景，这就是 Cognition 目前合并 PR 的吞吐量情况。大约始于 12 个月前，所以大家看到的大致是一年以来的快照。而在过去这一年里，这里蓝色的图表，这个 40 倍，代表我们工程团队产出的合并 PR 数量增长了 40 倍。现在大家可能也注意到了，底部有一条很细的黑线。它很容易被忽略，但那个 2.7 倍正是我们同期工程人员规模的增长。所以我们一直在招聘工程师，但我们每位工程师现在的生产力是以前的 10 倍以上。而且我敢说，这些 PR 的质量即使没有更高，也绝不亚于一年前。因此，我今天真正想探讨的核心就是：我们究竟做了什么，才释放了这 40 倍的工程吞吐量？从代码库到底层工程团队的组织方式，我们做了很多事情，我认为如果大家想复制同样的结果，这些都是非常实用的落地步骤。

英文原文

chief product officer at Cognition. We build a AI product called Devon. I want to spend most of my time here today though kind of abstracting a bit away from that and and focusing on what are we doing internally as an engineering team to move way faster. And so the backdrop I would put for all this is this is what our merged PR throughput looks like at Cognition. Uh the start of this is around 12 months ago. So you're looking at about a snapshot of a year. And over this last year, this blue chart here, this 40x is a 40x in the number of merged PRs we are producing on our engineering team. Now you might also notice there's a small black line along the bottom there. It's easy to miss, but that 2.7x that is our engineering headcount growth in the same period. So we have been hiring engineers, but each of our engineers is now more than 10x more productive than they were before. And I would dare to say that these PRs are just as high quality, if not higher quality, than they were a year ago. And so really the crux of what I want to talk about today is what did we do to unlock this 40x engineering throughput. And there's a lot of things we did from both the code base to the way we structure engineering teams that I think are very practical steps you can take if you want to reproduce the same results.

大家很多人可能都熟悉软件工厂。你们当中有多少人听说过软件工厂？让我看看有多少人举手。太棒了。那么，我会先从基础讲起，然后再逐步深入到更进阶的内容。软件工厂的基本理念就是无需任何人类参与。不过很多时候仍然需要人类的判断，那时我们就会引入升级上报机制，但默认情况下，我们的整个软件开发流程端到端全由 Devon 运行。

英文原文

Many of you are probably familiar with the software factory. How many of you guys have heard of software factory? Let's see a raise of hands. Cool. So, I'll start with the basics and then I'll work my way up to the more advanced stuff. The basic idea of a software factory is without a single human involved. Oftent time human judgment is needed though and that's where we have escalations but by default our entire software development process is just run by Devon from end to end.

这意味着的一件事就是，软件工程师的角色必须发生相当大的改变。大约在今年 3 月，我们与工程团队进行了这样一次对话，我们说：“听着，如果你现在只是从 Linear 任务看板上认领工单，然后发给智能体去实现，那么在 AI 时代你真的很难取得成功，因为任何能够被描述且能够被实现的任务，我们都能交给 Devon。” 所以说，每个人都必须改变自己的研发方式。而我们推行这一转变的方式，就是将原来的大团队拆分成一个个更小的 pod。在这些小 pod 中，工程师们不仅能获得成群的智能体替他们工作，还被赋予了更多的垂直业务职责。

英文原文

One of the things this means is that the role of the software engineer has to change quite a bit. Around March of this year, we had this conversation with the engineering team and we said, "Look, if you are right now just picking off tickets off a linear board and then sending that to an agent and implementing it, you're not going to really be set up for success in the age of AI because anything that is described and can be implemented, we'll be able to send to Devon." So really, everyone has to change the way that they're developing. And the way we've enforced this is we've taken our larger teams and we've broken them up into smaller pods. In these smaller pods, not only are given swarms of agents to do work for them, they are given more vertical responsibility.

因此，一个团队在某一时刻可能只有两到三名工程师，但他们同时还要负责工程开发之外的一切事务。他们要负责倾听客户的 Bug 反馈、进行 QA 测试，还要承担 PM 工作与产品管理职责。基本上，每位工程师都在管理自己一整套待办事项的 Linear 看板，并将任务派发给智能体替他们实现。所以，这确实是我们工程师工作方式上的一次根本性变革。这是我们过去一年中推行的举措之一，同时我们也在帮助许多大型组织自行落地这一模式。

英文原文

So instead, you might have only two or three engineers on a team at a time, but they're responsible for everything beyond the engineering as well. They're responsible for listening to customer bug reports. They're responsible for doing QA testing. They're responsible for doing the PMing and product management. And basically, every engineer manages their entire own linear board of things to do. and they send that to an agent to implement for them. So, it's really a fundamental change in the way that our engineers do work. This has been one of the things that we've implemented over the last year and we've been helping a lot of large organizations implement for themselves as well.

我还想重点说明的另一点是，我们全面在 Slack 中开展工作。举例来说，客户反馈的 Bug 报告大致是这样的：他们在 Slack 上给我发来一段视频，我们就直接把 Bug 报告转发到内部频道，@ 标注 Devon 并说：“嘿，请修复这个。” Devon 会立即响应，提交一份 PR，但同时还会附带发回一段带有修复和验证证明的视频。

英文原文

Another thing I'd really like to highlight is we work from Slack everywhere. So, this is an example of what a customer bug report might look like. They send me a video on Slack and we just forward bug reports straight into our internal channel and we tag Devon and say, "Hey, please fix this." Devon immediately comes up, shows up a PR, but then also sends back a video with the fix and proof.

这让我们的工作轻松了太多，因为在大多数工作中我完全不必离开 Slack。我们把所有的开发工作都抽象成了基于证据和视频，我可以直接把修复就绪的证明发回给客户，多数时候甚至连 IDE 或 GitHub 都不需要打开。

英文原文

And this makes our lives so much easier because I literally don't have to leave Slack for most of my work. We've abstracted away all the development to be based on evidence, videos, and I can just send the proof that the fix is ready right back to the customer without having open IDE or even GitHub most of the time.

所以，这就是大家最终希望达到的目标状态：即你可以在不打开 IDE、不打开任何一个编码智能体界面的情况下，完成整个开发生命周期。

英文原文

So, this is kind of the end goal you want to reach, which is a point where you can do your entire development life cycle without opening up an IDE or without opening up a single coding agent.

底层基础设施改造与本地沙箱化

但要达到这种状态，我们必须在底层做出大量的基础设施改造。对于任何想要实现这一目标的工程团队而言，这确实都是最硬核、最困难的部分。这绝不是单靠采购一个编码智能体就能解决的，你必须去改变自己的构建方式。以前在测试应用程序时，我们会针对云端部署进行测试。我们有预发环境，会连接到预发数据库，连接到线上活跃服务器，然后让工程师手动点击去端到端验证流程是否正常。

英文原文

But to get to this point, there's a lot of infrastructure we had to change underneath. And so this is the hard part that really every engineering team has to do if they want to get to that point. It's not just you purchase a coding agent. You got to go change the way you build. So before when we test our applications, we'd be testing them against cloud deployments. We'd have a staging environment, connect to a staging database, connect to live servers, and then we'd have our engineers manually go click through and verify that the flows work end to end.

我们进行了一次大规模迁移，使得每个服务都能通过 Docker Compose 在本地运行。针对每一个第三方服务，我们都制作了可以本地运行的 Mock 版本。这些全都可以在单个虚拟机内运行，并且每个 Devon 都能访问这些虚拟机。现在，我们公司内部的每一个编码智能体在测试代码变更时，完全不需要触及任何线上活跃服务。

英文原文

We did a big migration to move every service to be able to run locally with Docker Compose. We've made it so every third party service we have mocked versions to be able to run locally. And these can all run inside a single virtual machine that every Devon has access to. And every coding agent in our company now does not need to touch any live services at all to test their changes.

事实上，在公司内部我们每小时都会重建这些虚拟机。每隔一小时，我们就会拉取最新版本的代码。我们会从零开始全量重建整个代码库，以便能从零快速、轻松地运行起来。并且，如果由于代码库迭代过快导致这些构建失败，我们同样会让 Devon 作为第一响应者。它们总能立即着手修复这些虚拟机快照。因此，每当我们从 Slack 启动一个智能体时，它访问的都是一台全新的机器，运行着带有最新更新的全新版本代码库。顺便提一句，对于这些智能体，你也必须教会它们如何去测试你的代码库。不过使用云端编码智能体的好处在于，它们能够自我维护其训练闭环。在刚才提到的测试部分中，Devon 会自己编写关于如何测试我们代码库的技能。代码库会随时间演进，测试它的方式也会发生改变，但 Devon 能够不断更新这些技能，从而一次又一次地对其进行测试。这样带来的最终结果就是，大家真的再也不需要打开笔记本电脑去测试任何代码变更了。事实上，Cognition 的好几位工程师已经完全转向使用 iPad 来进行日常开发流程了。

英文原文

They in fact these VMs, we rebuild them every hour at the company. Every hour we pull the latest version of the code. We rebuild the entire codebase from scratch to be able to run very quickly and easily from scratch. Um, and if these builds ever fail because your codebase is rapidly changing, uh, we also have Devon be the first response there. And they're always immediately fixing these VM snapshots. So every time we spin up an agent from Slack, it goes to a fresh machine on a fresh version of the codebase with the latest updates. Um, and by the way, these agents, you need to teach them how to test your codebase as well. But the benefit of using cloud coding agents is that they can self-maintain their training loops. So that last section there on testing, Devon will write its own skills for how to test our codebase. The codebase will change over time. So the way you test it will change, but Devon can keep on updating these skills so that tests it over and over again. So the end result of this is literally people do not need to open their laptops anymore to test any code changes. In fact, several engineers at Cognition have already moved to using iPads for their development process.

顺便说一下，iPad 也更安全。大家知道，最近看到了很多安全入侵事件，其中很大一部分是因为 DevOps 以及人员泄露秘钥造成的。谁知道开发者的机器上都装了些什么？你的电脑里可能到处都散落着各类敏感凭证。而当一切都置于云端这些受控的虚拟机中时，你的安全状况实际上要好得多。

英文原文

By the way, iPads are also more secure. You know, a lot you've seen a lot of these hacks recently. A lot of it is because of DevOps and and people leaking keys. Who knows what's on your developer machine? You might have secrets all over the place. Uh when everything is in the cloud in these controlled VMs, you're actually in a much more secure spot.

我已经展示过一些可供测试的基础场景。我们还测试过高级的多人协作配置，这非常酷。以往在开发 Devon 接入 Zoom 通话进行语音交流的功能时，我们实际上会让测试中的 Devon 自行创建多个账号，让它们全部从同一台虚拟机内加入同一个 Zoom 会议，并来回发送语音备忘录，以确保功能正常运行。因此，如果你真正挖掘代码库的极限，就能实现相当高级的配置方案。

英文原文

Um I've showed some basic versions of things you can test. We've also tested advanced multiplayer setups, which is really cool. We've oftent times when we were developing Devon's ability to join voice in Zoom calls, we actually had Devon when it tests create multiple accounts, have them join the same Zoom call all from within their one VM and send voice memos back and forth to make sure that the functionality is working. So you can do quite advanced setups if you really push the limits of your codebase.

递归自我优化机制

而在这一切之上，大家显然都想要实现递归式的自我优化。我认为大家往往大多是从模型层面来考虑这一点的，但实际上大家细想一下，如果配置了云端智能体，你整个代码库中编写的每一行代码、生成的每一个 PR、做出的每一项决策，全都会被完整追踪记录下来。既然如此，完全没有理由不在此基础之上也运行智能体，去执行数据分析，并找出我们究竟在哪里浪费了时间、在哪里浪费了 Token、哪些环节可以做成脚本，以及我们能够添加哪些测试来改进这一流程。因此我们设立了一个专属频道，Devon 每天都会在里面发布想法，比如：“嘿，如果我们更改 Node 配置，大概能帮 CI 节省大约一分钟时间。如果我们修改这个 Lint 脚本，在这些 Bug 进入 AI 代码审查之前，我们大概能多拦截 20%。” 这确实带来了巨大的突破，你只需要搭建好这些系统，智能体就会主动去为你优化整个软件工厂。这直是

英文原文

Um, and actually on top of all this, you know, you obviously want recursive self-improvement. I think people think about this from the model sense quite a lot, but actually if you think about it, every single line of code that's being written, every single PR that's being generated, every decision, if you have a cloud agent set up, that's all tracked across your entire codebase. So there's no reason why you shouldn't also be able to run agents on top of that, do the data analysis, and figure out where are we wasting time, where are we wasting tokens, what, you know, things can be turned into scripts, what tests can we be adding to improve this process. And so we have a channel where Devon every day posts ideas about, hey, if we change our node setup, we can say shave like a minute off of our CI. If we change this lint script, we can like catch 20% more of these bugs before they ever hit AI code review. Um, and this has actually been a huge unlock where you can just set up these systems and the agents will go and optimize your software factory for you. Um, this has been

非常令人兴奋。

英文原文

really exciting.

从派发任务到交付职责

我经常听不到大家讨论、但我认为确实是下一步发展方向的事情之一，就是从给智能体分配任务的世界，转向给智能体交付职责的世界。这可以说是一种范式转变。你真的必须开始把这些东西当作同事来对待。那么，给智能体交付职责看起来是什么样的？假设我关心后端性能。

英文原文

One of the things that I don't hear people talking about a lot, but I think is really where things are going next is moving from the world where you're giving tasks to your agents to giving jobs to your agents. And this is a bit of a paradigm shift. You really have to start treating these things like co-workers. So what is giving a job to an agent look like? Imagine I care about back-end performance.

实际上，该特性的开发流程应该是这样的。你应该发送给你的智能体：“嘿，我想优化我们应用程序的 P99 延迟。”它不该是由你告诉它该怎么做来实现这点，而是应该说：“好的，P99 延迟，让我来添加遥测指标，弄清楚我们今天的 P99 延迟是多少。让我设置警报，以便我能弄清楚我们未来需要达到什么水平。让我去拆解我们为了能够改变这个延迟所需要做的每一个具体任务。让我创建 Datadog 警报。让我每周在 Slack 中生成报告。并且让我每天在你的手机上给你发送更新，汇报 P99 延迟目前是多少，以及改善了多少。”

英文原文

Really, this is what the development flow of the feature should look like. You should send to your agent, hey, I want to improve P99 latency of my application. Instead of you telling it what to do to do that, it should say, okay, P99 latency, let me add the telemetry to figure out where our P99 latency is today. Let me set up alerts so I can figure out where we need to be in the future. Let me go break down every single task we need to do to be able to move this latency. Let me create data dog alerts. Let me create weekly reports in Slack. and let me send you updates on your phone every day on here's where P99 latency is and here's how much it's improved.

从根本上说，这就是我们在 Cognition 鼓励所有工程师构建的系统类型。大多数时候我们甚至都不希望你去提示你的智能体。我们希望你只给你的智能体设定它们可以在较长时间跨度内为之努力的目标。而且你公司里任何你认为不可衡量的事情，想办法去衡量它。比如，你关心应用程序的性能吗？去测量一下你在应用程序上的点击延迟。如果你的应用程序有很多奇怪的布局偏移，去测量外接矩形框，并尝试找出如何尽量减少整个应用程序中的布局偏移。显然有 P99 延迟，但也有类似入职留存率这样的事情。

英文原文

This is fundamentally the type of systems that we encourage all of our engineers to be building a cognition. We don't even we don't even want you to be prompting your agents most of the time. We want you to just be giving your agent goals that they can work toward over long durations of time. And anything in your company that you think isn't measurable, find a way to measure it. You know, do you care about your application performance? Go measure, you know, the latency of clicks on your application. If your application has a lot of weird layout shifts, go measure the bounding boxes and try to figure out how you can minimize layout shifts across your application. There's obviously P9 latency, but there's also things like, you know, onboarding retention.

你们作为工程师所做的工作中，有多少是可以归纳为一个单一数字或指标的？我鼓励你们，如果你能做到这一点，你就应该直接设置一个智能体去 7×24 小时全天候优化那个数字。

英文原文

How much of what you guys do as engineers is things you can boil down to a single number or metric. And I encourage you that if you can do that, you should just set up an agent to go optimize that number 247.

案例：GPU 推理延迟与安卓应用开发

关于这一点的一个极佳例子就是推理延迟。在 Cognition，我们很幸运成为英伟达新一代 Vera Rubin GPU 的首批接收者。

英文原文

So a great example of this is inference latency. At Cognition, we're lucky to be the first receivers of Nvidia's new class of Vera Rubin GPUs.

在我们收到这些 GPU 时，内核还没有为它们构建好。因此我们必须从零开始摸索如何在这些即将推出的新硬件之上构建我们自己的推理栈。智能体让我们能够以比以往高得多的效率来完成这项工作。所以我们只是告诉它：“嘿，我们关心这些 GPU 的吞吐量或 TPM。去弄清楚如何将其推向极限。”然后在大约……我想这里大概是一周的时间里，智能体能够去——我们给它分配了一组固定的 GPU 供其开展工作。它能够去寻找不同的策略并测试它们。对于它发现效果最好的策略，它会发送给人类工程师进行最终批准。但基本上大约有八九种不同的事情是通过直接给这些智能体这些高层目标而被大致发现的，我们得以将这些新的 Rubin GPU 的吞吐量从比新的 GB300 更差的状态，提升到如今明显优于上一代 GB300 GPU 的水平。同样，这一切都只是通过给你的智能体一个最终目标来完成的。

英文原文

Kernels were not built for these GPUs or we received them. So we had to from scratch figure out how do we build our own inference stack on top of these this new hardware that's coming out. And agents have allowed us to do this in a way that is much more efficient than before. So we just tell it, hey, we care about the throughput or TPM of these GPUs. Go figure out how to push that to its limits. And over the course of what is about, I guess here like a week, um the agent was able to go, we we allocated it a permanent set of GPUs to work on. It was able to go find different strategies, test them. Uh the strategies that it found worked best, it would send to a human engineer for final approval. But basically there's eight or nine different things that were roughly discovered via just giving these agents these high level goals and we were able to push uh the throughput on these new verbin GPUs from what was worse than the new GB300s to what is now significantly better than the previous class of GB300 GPUs. Um and again this is all done by just giving a end goal to your agents.

另一个我喜欢和大家分享的例子，是我们的工程团队规模相当小。这意味着我们往往没有足够的带宽来为我们可能想做的事情配备充足的人手，比如安卓应用程序。

英文原文

Another example that I like to uh share with folks, we we are a fairly small engineering team. It means we do not have oftentimes the bandwidth to fully staff things that we might want to do like an Android application.

但是前几天，我们团队的一位工程师 Sam，他有了这个绝妙的想法，他说：“好的，看，我们有一个团队的人正在致力于构建一个出色的 iOS 应用程序。我们难道不能直接把那个当作我们安卓应用程序的蓝图，然后告诉 Devin：‘嘿，去端到端地维护我们从 iOS 应用程序到安卓应用程序的对齐’吗？”而这正是他所做的。但这不仅仅是说，嘿，开一次 Devin 会话去搞定这一切。他说的是：设置常驻的钩子对接我们的代码库以监控 iOS 应用程序；去监听 Slack 频道中用户对我们应用程序的反馈，然后递归地改进我们的应用程序；去对每一个细节进行实时的 QA 测试并确保一切都能运行。实时同步这些东西真是太酷了。我认为最终的结果实际上相当棒。

英文原文

But the other day, one of our engineers on our team, Sam, he had this brilliant idea is he said, "Okay, look, we have a team of people working on building a great iOS application. Can't we just use that as a blueprint for our Android application and tell Devon, hey, go maintain parody with from our iOS application to our Android application end to end. And that is exactly what he did. But it wasn't just, hey, one Devon session to go do all this. He said, set up permanent hooks against our codebase to monitor the iOS application. uh you know go listen to the Slack channels for user feedback about our applications and then go recursively improve our application, go do live QA testing of every detail and making sure it all works. Um syncing these things live has been really cool. Uh and I think the the final results here are actually quite sick.

所以我们真的就只给了我们的智能体这一个高层提示，那就是：“嘿，去管理整个安卓应用程序。”在 44 天的时间里，我们拥有了一个可运行的应用程序，我们很快就会发布它。但是作为一段延时缩影，大多数日子里没有一条人类消息发送给这个正在处理该任务的智能体，而在那些日子里，它每天合并多达四五十个 PR。所有这些都针对 iOS 应用程序进行了像素级的实时 QA 测试，测试是否……在一天结束时仅将最重要的决策升级汇报给人类。

英文原文

So we really just gave our agent this one highle prompt which is hey go manage the entire Android application and over the course of 44 days uh we have a working application which we'll be putting out soon but as a time lapse most days there's not a single human message going to this agent working on it and on those days you know it's merging upward of like 40 50 PRs a day all this is being live QA tested against the iOS application pixel for pixel uh testing whether you know escalating only the most important decisions to a human at the end of the day

而且大多数时候我们的工程师甚至都不需要打开 Devin，他们只是从 Slack 中查看它发回给他们的录屏。所以在最后，人类在这个闭环中几乎什么都不用做，这就是在整个过程中人类实际不得不发送给我们智能体的消息数量。

英文原文

and most of the time our engineers don't even need to look open devon they're just from Slack looking at the recordings that it sends back to them so in the end the human is barely doing anything in this loop this is how many messages a human actually had to send to our agent in this entire process.

是的，它不是粗制滥造的产物，它不是一个垃圾应用程序。这实际上是我们很快就会上线发布的东西。

英文原文

Um, yeah, and it's not a slop it's not a slop application. Uh, this is actually something that we will be going live with very soon.

好的，太棒了。这些实际上就是我要分享的全部内容了。我认为从所有这些当中得出的最终信息是，如果你正确地为智能体设置了闭环，你就能做出一些非常不可思议的事情。所以我认为大多数团队其实是被他们拥有的雄心壮志所限制了。所以是时候雄心勃勃了。谢谢大家。

英文原文

So, cool. Uh, that's actually all I had. Um, I think the end message from all of this is if you set up the loops for agents correctly, you can do some pretty incredible stuff. So I think really most teams are limited by the amount of ambition they have. So it's time to be ambitious. Thank you.

[掌声]

英文原文

[applause]

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Hey, hey, hey.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

英文原文

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Hey, hey, hey.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

智能体与网络搜索

好的。大家好。我叫 Jeff Pinner。我是 Exa 的 CTO。感谢各位邀请我们来到这里。我们今天打算稍微聊一聊你的智能体所不知道的事情。

英文原文

All right. Hello everyone. My name is Jeff Pinner. I'm the CTO of Exa. Thank you guys for having us here. Uh, and we're going to talk a little bit today about things your agent doesn't know.

网络搜索到现在已经存在了大约 30 年，时间长到在现阶段我们对此已经习以为常，而且我们并不会真正去质疑现状，或者质疑它在智能体时代可以有何不同。

英文原文

So, web search has been around now for about 30 years, long enough where at this point we're kind of used to it and we don't really question the status quo or how it could be different with agents.

在过去大约 18 个月里，搜索迎来了智能体作为一种新的客户群体。它们在代表我们进行搜索。因此，在 Exa，我们专门将网络搜索构建为面向智能体的知识层。我们的请求量已经从两年前的每月约 1250 万次，增长到了刚刚过去的 10 月份的略高于 13 亿次。所以在两年间增长了约 100 倍，而且这并不是 Exa 独有的现象。如果你去交流，OpenAI 刚刚宣布 ChatGPT 处理的搜索量大约是谷歌搜索量的五分之一，而且智能体总体上现在占到了网络流量的大约一半。因此，我认为一个很自然的问题是，为什么智能体到底需要搜索？所以在我们讨论我们如何为智能体做搜索之前，让我们先尝试回答这个问题。

英文原文

Over the last 18 months or so, search has picked up agents as sort of a new customer. They're searching on our behalf. And so, at EXO, we specifically build web search as a knowledge layer for agents. We've gone from about 12 and a half million requests a month two years ago to a little over 1.3 billion this past October. So about 100x over two years and it's not unique to EXA. If you talk OpenAI just announced that ChatGBT handles about a fifth of Google search volumes and agents in general make up just around half the volume of web traffic now. And so I think the natural question is why do agents need search at all? So let's start with trying to answer this before we talk about how we do search for agents.

前沿模型可以依靠参数化知识完成很多任务。

英文原文

So frontier models can do a lot of tasks from parametric knowledge.

但遗憾的是，对于现实世界的任务来说，这根本不够。通常前沿模型无论功能多么强大，都总是过时的。模型发布从训练截止到正式推出需要两到五个月的时间。然后在此之后通常还要经过几个月，下一个前沿模型才会发布。因此综合来看，无论最新的知识库是什么，你都落后大约四到八个月。与此同时，世界并不会仅仅因为一个模型从训练阶段转入推理阶段就停止变化。代码库发布、API 变更、价格变动、新法规出台、法院判决产生。世界在继续前行。

英文原文

But unfortunately for real world real world tasks that simply isn't enough. Generally frontier models no matter how capable they are are always out of date. Model launches take two to five months from training cuto off to launch. Then there's generally a few months after that before the next frontier model is released. And so taking that together, you're about four to eight months out of date with whatever the latest knowledge base is. And in the meantime, the world doesn't stop changing just because a model has moved from training to inference. Libraries launch, APIs change, prices change, new regulations appear, court rulings happen. Um, the world continues to go.

而且即使事实并非如此，训练数据本身实际上也只是模型所接受训练知识的一小部分。

英文原文

And even if that weren't true, training data itself is actually just a small amount of knowledge that the model is trained on.

目前最大规模的模型是在大约 50 万亿个 token 上训练的。网络本身大约有 3100 万亿个 token。这还包括诸如登录系统和应用程序背后的深层网络。如果你将其缩小到仅被建立索引的网络，也就是搜索引擎能够看到的部分，那仍然大约有 510 万亿个 token。这意味着即使在模型截止发生后世界没有继续演变，你仍然……

英文原文

The biggest models out there are trained on around 50 trillion tokens. The web itself is around 3,100 trillion. And that includes the deep web behind things like login and apps. If you focus it down to just the web that is indexed, the thing that search engines can see, that's still about 510 trillion tokens. Which means even if the world didn't continue to evolve after the model cut off happened, you would still

……只索引了模型必须去寻找的外界知识中极小的一部分。

英文原文

only be indexing a very small amount of the knowledge that's out there that models have to go and find.

而对于我们希望模型去做的任务类型，希望智能体去做的任务类型，有许多基于外部知识的有价值工作正在开展。无论是在编程、法律工作，还是尽职调查工作中，通常所有这些任务都需要跨多个来源获取正确且完整的知识。因此，这正是我们试图弥合的差距。

英文原文

And for the type of tasks we want models to do, we want agents to do, there is valuable work that happens that is based on outside knowledge. Whether it's across coding, legal work, due diligence work, generally all these tasks require correct complete knowledge across multiple sources. And so this is what we're trying to the gap we're trying to close.

Atlas 基准与模型检索表现

现在，在尝试弥合任何差距时，首先要做的事情就是尝试对其进行衡量。因此，我们做的第一件事就是创建了一个名为 Atlas 的基准，用来测试评测架构（harnesses）利用现实世界知识的能力。

英文原文

Now, the first thing to do when we're trying to close any gap is to try to measure it. And so, the first thing we do is create a benchmark called Atlas, which tries to test how well harnesses can leverage real world knowledge.

所以，我们创建了 Atlas 基准来尝试在这些任务中测试智能体。

首先，我们要求智能体找到符合条件的实体。

然后，我们要求智能体利用它们所能找到的尽可能多的信息来丰富这些知识。

最后，我们对照标准答案（golden answer key）给它们评分。

我们只将正确识别出我们寻找的所有维度的行计为正确。因此，要在 Atlas 上取得成功，你需要始终保持正确，而不仅仅是对单次搜索拥有不错的命中率。

英文原文

So, we created the Atlas benchmark to try to test agents in these tasks.

First, we require agents to find entities that match a condition.

Then, we require agents to enrich the knowledge with as much information as they can find.

And finally, we score them against the golden answer key.

And we only count rows correct that correctly identify all aspects of what we're looking for. And so to succeed at Atlas, you need to be consistently right and not just have a decent hit rate for individual searches.

因此，我们将以此作为衡量智能体在知识检索方面表现的基准。我们发现的第一件事是，模型记忆会随着时间推移（新近度）而衰退。

我们在 Atlas 上测试智能体时看到，模型成功率一旦遇到训练集未覆盖的数据，就会急剧下降。事实越新，模型知道的就越少。而模型需要——或者我应该说智能体需要——数据来尝试完成这些任务。因此，我们首先能看到的是，仅靠记忆、仅靠参数化记忆（parametric memory）是不够的。

英文原文

So we're going to use this as a benchmark for how we measure agent performance when it comes to knowledge retrieval. And the first thing that we find is that model memory falls off with recency.

When we test agents on Atlas, we see that model success rates fall off sharply as soon as they enter as soon as they encounter data that's not covered in the training set. The newer the fact is, the less the models know. And models require or I should say agents require data to try to achieve these tasks. And so the first thing that we can see is that memory alone, parametric memory isn't sufficient.

因此，当我们运行仅依赖参数化记忆而没有网络搜索的智能体时，它们在 Atlas 上的得分在 9% 左右。所以我们将以此来尝试定量理解网络搜索是如何提升性能的。

英文原文

So when we run agents with just parametric memory without web search, they score around 9% on Atlas. And so we'll use this to try to understand quantitatively how web search improves performance.

智能体搜索的三大挑战

那么接下来的一个自然而然的问题是：为什么它们不直接去 Google 搜一下呢？[哧笑]

第一个原因是，智能体搜索的方式与人类不同。

通常人们在 Google 中输入大约 3 个词。呃，这比 6 年前的大约 2 个词有所增加。另一方面，智能体搜索时使用的或多或少是完整的句子。在 Exa 上的平均搜索查询大约是 12 个词。Exa 上大约 10% 的查询长达 17 个词或更多。

而这些词增加了限制条件、版本信息、日期、地点，以及各种不同类型的条件，这些都是基于关键字排名的系统原本并非为了处理这些条件而设计的。因此我们来看一个例子。

英文原文

So the next natural question is why don't they just Google it? [snorts]

And the first reason is agents don't search like people.

Generally people type about three words into Google. Uh, and that's up from about two words six years ago. Agents, on the other hand, search with more or less full sentences. The mean search on X is about 12 words. About 10% of queries on EXA are 17 words or higher.

And these words add constraints, versioning information, dates, places, all different types of conditions on the on the search that keyword ranking wasn't really designed to handle. And so we'll take a look at an example of that.

这是来自 Exa 的一个真实查询。它是一条寻找公司的请求。它是一个完整的句子，添加了一堆限制条件。当你把它输入到关键字检索算法中时，你真正得到的是寥寥几个根本没有回答查询所问内容的结果。呃，搜索并没有理解这个查询。其中第三个结果明确位于美国境内，而查询所要求的却是美国境外的数据。

几分钟后我们还会回过头来看这个。

英文原文

So this is an actual query from Exa. It's a request for companies. It's a full sentence. It adds a bunch of constraints. And when you feed this into a keyword retrieval algorithm, what you really get is a handful of results that don't really answer what the query asked for. Uh the search didn't understand the query. The result, the third one is specifically in the US and the result asks for data outside the US.

And so we're going to come back to this in a few minutes.

因为我想聚焦的第二件事是这样一个事实：对于网络上的许多知识，搜索根本无法触及。大多数有价值的信息并不在公开网络上。报道的新闻、人员数据、财务数据、研究，它们都处于付费墙之后。因此，抓取开放互联网的搜索引擎永远看不到这些页面。

英文原文

Because the second thing I want to focus on is the fact that for a lot of knowledge on the web, search just doesn't have access to it. Most of the valuable information isn't on the open web. Reported news, people data, financial data, research, they're all paywalled off. And so search engines that crawl the open internet don't ever see these fa pages.

最后，即使知识位于公开网络上且你的智能体可以访问它，还存在第三个问题，那就是效率。

你知道，如果一个智能体进行 10 次搜索，从该搜索中读取 5 个页面，从每个页面获取大约 16k 的原始 HTML，仅在处理结果上就大约需要 860,000 个 token。如果你还记得第一张幻灯片，我们每月大约处理 13 亿次请求。因此，仅仅为了提取信息就有大量的 token 在被消耗翻转。

英文原文

And then lastly, even when knowledge is on the open web and your agents can access it, there's a third problem which is efficiency.

You know, if an agent makes 10 searches, reads five pages from that search, gets about 16k of raw HTML from each page, that's about 860,000 tokens just in processing results. And if you remember from the first slide, we serve about 1.3 billion requests a month. So that's a lot of tokens that are just churning to extract information.

最后，当你加入关键字搜索并查看我们 Atlas 基准的情况时，得分确实有所提高。它补充了最新的知识，但我们只上升到了大约 28%。因为大多数行返回的数据依然是不完整的，或者干脆就是错误的值。

因此，当我们审视搜索时，面临的挑战是如何真正向智能体提供它们实际需要的知识。

英文原文

And then finally, when you add keyword search and we look at what happens to our Atlas benchmark, the score definitely improves. It's added recent knowledge, but we've gone up to about 28%. Because most of the rows still come back with incomplete data or just wrong values.

And so the challenge when we look at search is how do we actually provide the knowledge to agents that they actually require.

Exa 的语义检索与知识扩展解决方案

所以，我刚才提到我们会回到这个查询，至少在 Exa，我们尝试解决这项任务所做的第一件事就是从信息中提取含义，而不是仅做句法匹配或关键字匹配。

因此我们通读页面，理解查询的意图，在这里我们向你提供明确回答该意图并附带所有条件的结果。

英文原文

So the first thing I told you we'd come back to this query is at least at EXA the first thing we do to try to tackle this task is to extract meaning out of information as opposed to just syntactical matches or keyword matches.

And so we look through pages, we understand the intent of queries, and here we give you results that specifically answer the intent with all of the conditionals attached.

为了做到这一点，我们采取了针对现代网络爬虫的做法。我们爬取互联网，进行语义提取，将其嵌入到向量空间中，并建立索引。当你的查询进入时，我们将查询嵌入到同一个向量空间中，并找到最近的匹配项。这使我们能够从句法理解和关键字匹配转向对查询和网络内容的语义理解。

我们在高达每秒一百万页的处理量下执行此操作，呃，从抓取到建立索引的延迟大约为三分钟。

英文原文

And so in order to do this, we do what you would for a modern web crawler. We crawl the internet. We do a semantic extraction. We embed it in vector space. We index it. When your query comes in, we embed the query in the same vector space. and we find the nearest matches. And this lets us move from syntactical understanding and keyword matching to a semantic understanding of the query in the web.

And we do this at a volume of up to a million pages a second uh with a latency of about three minutes from crawl to index.

现在来看第二个问题，我们需要将其扩展到公开网络之外。

因此在这里，我们必须通过合作的专有数据以及我们提取和表示的知识图谱数据，来扩展公共网络上的内容。这样，智能体就可以检索到它针对自身问题所需的提问与答案，而无需知道去哪里寻找它们。这一切都在单个 API 之后完成。

英文原文

Now for the second problem, we need to extend it just beyond the open web.

And so here we have to extend what's on the public web with both proprietary data that's partnered and knowledge graph data that we extract and represent. And this way the agent can retrieve questions and answers that it needs to its questions without having to know where to look for them. Doing this behind a single API.

针对最后一个挑战，为了支撑智能体产生的查询量，我们不能简单地将它们引向网页。我们必须从页面中提取答案，并为智能体创建高 token 密度的表示形式以供处理。仅仅说明该查询的答案存在于这个网页上是不够的。

英文原文

And for the last challenge, in order to support the volume of queries that agents make, we can't simply point them to web pages. We have to extract the answers from the page and create a token dense representation for agents to process. It's not sufficient to say the answer to this query lives on this web page.

你可以拿谁因锂离子电池获得诺贝尔奖这个问题为例。如果只是指向维基百科页面，你将得到数千个字符。

实际对引用和溯源有价值的摘录大约只有 500 个字符。因此，当你必须直接处理来自 Google 的数据时，相比能够直接提取该数据的情况，涉及的 token 大约多出 16 倍。而且这些提取必须具有丰富的语义。你必须挑选出一段能够回答该问题的文本。因此，查询本身至关重要。

英文原文

You could take for example this question of who won the Nobel Prize for lithium-ion batteries. If you just point to the Wikipedia page, you'll get thousands of characters.

The actual excerpt that matters for citation and grounding is about 500. So there's about 16x more tokens when you have to process data directly from Google than when you can extract that data. And those extractions have to be semantically rich. You have to pick a segment of the text that answers the question. And so the query matters.

那么这究竟能给我们带来什么呢？

当我们再次使用 Atlas 进行衡量以确证自身时，我们看到我们已经在闭卷测试中从大约 9% 提升到了在整个互联网和扩展知识上进行语义提取时的大约 56%。

因此，我们不仅获得了大约 5 倍的分数提升，抱歉，而且由于提取带来的 token 效率，我们还降低了大约 28% 的成本。由此我们开始定量地证明，改善知识获取确实能提升底层智能体的性能。

英文原文

And so what does this actually get us?

When we measure with Atlas again to ground ourselves, we've seen that we've improved from about 9% on our closedbook test to about 56% when we do a semantic extraction over the internet and extended knowledge.

So not only do we get about 5x higher of a score, excuse me, but we also reduce the cost by about 28% due to the token efficiency of the extraction. And so we start to demonstrate quantitatively that improving access to knowledge actually improves performance of the underlying agent.

所以，呃，是的，总的来说，我认为我们希望能表达的主要结论是：虽然前沿模型的推理能力相对固定，但通过扩展外围架构并赋予其跨互联网访问语义的能力，可以在智能体性能方面取得很大提升。就这些。谢谢大家。

[掌声]

英文原文

And so uh yeah, in general I think the main takeaway that we've hopefully expressed is that while reasoning capabilities on our frontier models are relatively fixed, a lot of improvement can be done in aentic performance by extending the harness and giving it access to semantic meaning across the internet. That's it. Thank you.

[applause]

麦克风调试与亚马逊实践分享

Heat。Heat。

Heat。

Heat。

Heat。Heat。

Heat。Heat。

Heat。Heat。

Heat。Heat。

out。Hey

Heat。Heat。

Heat。

Heat。

Heat。Heat。N。

Hey，

hey，hey。

Heat。Hey, heat。Hey, heat。

英文原文

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

out. Hey

Heat. Heat.

Heat.

Heat.

Heat. Heat. N.

Hey,

hey, hey.

Heat. Hey, heat. Hey, heat.

好的，大家下午好。哇，声音真大。大家能听到我说话吧？看来能听到。

好的。我今天想先问一个问题。在座有多少人觉得他们今天已经从自己的智能体中获得了所能获得的一切，并且 AI 对你们团队的承诺已经兑现，而你们对自己目前的进展感到满意？请举手。

这和我平时看到的大致一样。几乎没有什么人举手。

英文原文

Well, good afternoon, folks. Wow, that's loud. You guys hear me? Okay, I take it.

Yes. I'm going to start with a question here today. How many people feel like they're getting everything that they could possibly get from their agents today and that the promise of AI for your team has been delivered and you're satisfied with where you are? Show hands.

This is about what I usually see. Like there's very few people.

对于我们在亚马逊的人来说，其实也是一模一样的。比如当我们刚开始这个过程时，情况是我们看到了 10% 到 20% 之间的增益，就像我在这边的图表上所展示的那样。你看，1.1 倍到 1.2 倍。2 倍。而我们注意到的是，当时发生的情况是各个团队把 AI 硬套进了他们原有的流程中。所以他们采用的是他们平时的常规流程，他们……

英文原文

And for us at Amazon, it's actually the very same. Like when we first started this process, what happened is we were seeing between 10 and 20% gains as I show on the chart over here. You know, 1.1x to 1.2x. 2x. And what we noticed was what was happening is teams were bolting AI onto a process that they already had. So they're taking their normal process they

他们之前就是这么做的。他们试图在这里加点 AI，在那里加点 AI，指望这样就能获得巨大的成效。此外，我们看到的是团队仅仅缩减了代码生成阶段。正如大家所知——你们大家在这方面都是专家——随着时间的推移，代码生成阶段已经变得越来越好。但是软件开发远不止代码生成而已。

英文原文

were doing. They're trying to attach some AI here, attach some AI there, and expecting that they're going to get this massive result out of it. In addition, what we saw was teams were only shrinking the code generation phase. And as you all know, you guys are all experts in this. The code generation phase has gotten better and better and better as time has gone on. But there's a lot more to software development than just code generation.

我们在 6 月做了一项研究。我——我在上面发表过一篇博文，或者说我们也据此发表了一篇博文。在研究中，我们观察了亚马逊零售部门（Amazon stores division）中改变了工作方式的团队。我们审视了他们在 AI 投资上获得了什么样的回报。许多这类团队在能够提速之前，速度其实是变慢了的。但是中位数团队在生产力提升方面看到了 4.5 倍的回报。具体来说，这是对照该团队自身历史基线归一化后的开发速度；我们研究中还有一些团队超过了 10 倍。对，所以我们谈论的是生产力显著提高，而他们所做的不过是改变了工作方式。但我仍然常常看到这种情况——从你们举手的情况来看也是如此，对吧——人们觉得 AI 的那种承诺似乎还没有完全兑现。

英文原文

We did a study in June. I I've published a blog post on or we published a blog post on it as well. And in the study, we looked at teams who changed how they worked in our Amazon stores division. And we we looked at what kind of returns were they getting on their AI investment. Many of these teams were going slower before they were able to go faster. But the median team was seeing a 4.5x return in in terms of productivity improvement. And specifically what this was was normalized development velocity against that own team's historical baseline and we had some teams in the study who exceeded 10x right so we're talking significantly more productive and what they did was just they changed the way they were working but I see this still so often and I just by your show of hands the same thing right where people don't feel like that promise of AI has quite been delivered yet

我叫 Doug Clawson，是 Curo 团队的产品经理，从一开始我就真正参与了这次 AI 开发者工具的历程，而我仍然像我说的那样，看到有些团队没有完全得到他们预期的东西。

英文原文

now my name is Doug Clawson I'm a product manager on the Curo team and I've been on this AI dev tools journey really since the beginning and I still see like I said teams who aren't quite getting what they expect.

现在，作为这方面的一名 PM，我基本上一直被问到同样的两个问题，这就像是我人生和会议中的家常便饭。其一是“你们在发布什么？”他们想知道具体有哪些新功能，那些新功能可能会开启哪些新的用例。你们知道，我可以一一介绍这些，那很好。另一个问题——通常也是更有趣的问题，因为每个人构建软件的方式都略有不同——是“我们这样做对吗？”就好像大家觉得自己陷入了某种局部最优，并且担心其他人已经摸索出了比他们更好地使用 AI 的方法，自己马上就要被超越了。我们今天实际上要花时间讨论的是这里的第二个问题，而不是第一个问题。我将带大家了解我们其中的一个团队——我们的 Curo crew 团队——如今是如何每周发布成千上万个 PR 的，我们在过程中学到的重大教训，我们走到今天所经历的大致过程；然后在这场结束时，我会给大家留一个工作坊，如果您想尝试在 Kuroko crew 中构建自己的软件工厂，也可以去做。

英文原文

Now as a PM on her I get asked literally the same two questions all the time like this is a story of my life and meetings. What are you shipping? And they want to know like what are the new features? What new use cases might be enabled by those new features? And you know I can walk through that and that's great. The other question, and oftentimes the more interesting one because everybody builds software slightly differently, is are we doing this right? Like people feel like they've hit some sort of local maximum and they're scared that somebody else has figured out how to use the AI better than they have and they're just going to get surpassed. We're actually going to spend today on the second question here, not the first question. And I'm going to walk you through how one of our teams, our Curo crew team, is shipping now thousands of PRs per week. the big lessons we learned on the way and sort of the process we went through to get to the where we are today and then at the end of this I'm going to leave you with a workshop that if you want to try to build your own software factory in Kuroko crew you can go and do that as well

不好意思，[清嗓子]。首先，先简单介绍一下 Kuro 的背景。Kuro 是一个 Agent 化的开发系统，它让你可以利用 AI Agent 来规划、构建和交付软件。它在你工作的任何地方都能运行，所以在这里我向大家展示的，我们有一个 IDE 客户端、一个 CLI、一个网页端和一个移动端客户端，然后刚才我也简要谈到了 Curo crew。我们支持多种模型。所以在 Kuro 中，我们既支持 Anthropic 模型，也支持 OpenAI 模型，以及一些开源权重模型。稍后我会谈谈我们如何使用它，以及它对我们有什么好处。

英文原文

excuse me [clears throat] now first off just some bit of context on Kuro so Kuro is an agentic development system that lets you plan build and ship software with AI agents it works wherever you work so up here I'm showing you we have an IDE client a CLI a web and a mobile client and then I just talked Just briefly, we have Curo crew as well. Um, we support a variety of models. So, we have both the anthropic models and the open AI models in in Kuro as well as some openweight models. I'll talk a little bit later how we use that and how that's been beneficial to us.

就像 AWS 的其他部分一样，我们确实在不断突破性价比的界限。因此，当我们开始节约资金、让模型运行成本更低、让 Agent 运行成本更低时，我们最终会把这些节省下来的成本传递给你们。所以，对于 Kira 在性价比和成本效益这一层面所达到的状态，我们非常满意。

英文原文

And just like the rest of AWS, we really push the barrier on cost performance. So, when we start saving money and we make the models cheaper to run and the agents cheaper to run, we end up passing those savings on to you. So, we're really happy with where Kira has gotten on that paro front of sort of cost performance and cost effectiveness.

Kuro Crew 的演进

不好意思。[清嗓子] 就像我说的，我们今天要把时间花在讨论 Kuro Crew 上。Kira Crew 打造出来的目的就是同时运行许多 Kuro Agent。它完全是开源的。它实际上最初是在亚马逊内部作为一个内部产品开始的，当时一些工程师拿了我们的 CLI，他们觉得：“嘿，我们想扩展这个。”那是在整个 Open Claw 热潮期间发生的。如果在座有人家里有 Mac mini，他们一直在跑那些东西。它就发生在那股热潮期间，当时人们想弄清楚怎样才能让这些 Agent 更像个人助理。于是内部开发出了这个 Meshclaw 产品，在前两个月就吸引了大约 40,000 名新加入的亚马逊内部开发者使用。因此，作为 Curo 团队，我们觉得既然它是构建在 Curo 之上的，我们就应该吸收它、进行扩展并把它发布出来。现在我们已经做到了这一点。Kuro Crew 已经发布了，它是开源的。谈到我们在 Curo crew 团队以及我们团队在构建和开发这个产品的过程中看到了什么——

英文原文

Excuse me. Now, [clears throat] like I said, we're going to spend the day talking about Kuro Crew. And Kira Crew was built to run many Kuro agents at the same time. It's all open source. It actually started internally as an internal Amazon product where uh some engineers grabbed our CLI and they're like, "Hey, we want to extend this." It was during the the whole Open Claw craze. If anybody is has their Mac minis at home, they've been running those. It happened during that craze where people wanted to figure out how they could get these agents to be more like personal assistants. And so this Meshclaw product internally was developed, got around 40,000 new uh internal Amazon developers on it in the first two months. And so us as the Curo team, we're like, hey, we should since it was built on Curo, we should take that and extend it and release it. And so we've done that now. And Kuro Crew is released. It's open source. Uh and if I talk about what what have we seen with the Curo crew team and and our team in terms of building and developing this

我们在 8 月 4 日向外部发布，同样，它是开源的，所以你可以去查看所有的源代码，查看提交上来的所有 PR。我们遵循一条理念：如果我们自己不用它、不喜爱它，我们就不发布它。自推出以来，我们凭借一支非常小规模的团队合并了大约 16,500 个 PR。就在上周，仅单周就超过了 3,000 个。我们接受来自外部实体的贡献，既有亚马逊内部的，也有亚马逊外部的，就是那些想要贡献、并借助开源带来好点子的人。

英文原文

we we shipped on August 4th externally and again it's open source so you can go look at all the source and look at all the PRs that came through. We go by one philosophy that if we don't use it and love it then we don't ship it. Since we launched we've merged about 16,500 PRs with a very small team. Uh just last week there was over 3,000 in a single week alone. take contributions from external uh entities to both internal to Amazon and then external to Amazon, people who just want to contribute and take a good idea with open source.

如果你看一下这里的图表，我们大致稳定在——你知道，如果只看我们的发布速度，在发布后的前几个月，我们大致稳定在几百的高位，600 到 800 左右，然后你会看到比如在 928，对吧？突然之间我们再次突破了一千，然后大致在徘徊。我在脑子里平均一下，就算 1.2K 左右，对吧？然后你可以看到我们又实现了一次阶跃式的提升。虽然我在这张图表上给你们展示的是 PR，但团队内部真正关心的并不一定是 PR，我们关心的是单个开发者同时能够操作多少个会话。我想明确一点，这并不是一套放之四海而皆准的东西。因此，在我讲解的过程中，请记住这一点：对我们有效的方法可能对你们并不适用。所以你们应该审视，你们正在做的事情中，有哪些是可以从这里借鉴或学习的？

英文原文

And if you look at the chart here, we sort of leveled out at around, you know, if you look at just our shipping speed, we sort of leveled out at around uh high hundreds, 600 to 800 sort of the first few months we after we shipped and then you'll see in like 928, right? All of a sudden we break a thousand again and then we're sort of hovering around. I'll average it out in my head and say 1.2Kish, right? And then you can see we made another step function improvement. And while I'm showing you PRs on this chart, what we really care about among the team is not necessarily the PRs, but we care about how many sessions can an individual developer operate at the same time. And I want to be clear that like this is not a one-sizefits-all thing. So as I walk through this, just keep that in mind that that what works for us may not work for you. And so you should challenge, you know, what are you doing that you could take or you could learn from this?

关于这一点最后要说的是，这些里面大约每 10 个中有 1 个是功能。所以我总被问到：天哪，3,300 个功能，一周里可真够多的。其实并不是，并不全都是功能，对吧？快速交付就是这个样子的。很多人问我，好吧，如果大约每 10 个中就有 1 个是功能，也就是大约 10%，那么如果 10% 是功能，另外 90% 是什么？对我们来说是测试，测试是我们工作很大的一部分；是对我们已经发布的东西进行一些所谓的小痛点修复（paper cut fixes），也就是我们刚开始了解它是怎么被使用的，并对其稍作调整；此外还有 bug 修复。这就是高速度、快节奏发布真正呈现的样子，对吧？它并不全都是新功能，但只要想想我们那 10% 的功能，在这个时间跨度内依然交付了数百个新功能。

英文原文

Uh, last thing on this one, about one in 10 of these is features. So I always get asked like, man, 3,300 features, that's a lot of features in a week. Well, no, it's not all features, right? This is what shipping with speed looks like. Um, a lot of So I get asked, okay, if so about one in 10 of these is a feature, so about 10%. And so if 10% are a feature, what are the other 90%. Well, they're for us, it's tests. test is a big part of what we do. It's uh I'll say paper cut fixes for things we've already shipped that we're just sort of learning about how it it's getting used and we're massaging it a little bit. Uh it's bug fixes as well. And that's really what shipping at a high speed with velocity looks like, right? It it's not all new features, but if you just think about 10% of our features, there's still hundreds of features, new features being shipped over this timeline.

如果想一想我们的开发周期是如何演进的？我们最初——如果我回溯到我们的 CLI 产品最初推出的时候，是单会话、单人模式。我相信当我们这些新 Agent 刚出来的时候，我们许多人都经历过这种情况：我打字，然后它工作，接着它停止工作，然后我再打字，对吧？我们就是这样来来回回，所以那是单会话，对吧？我说过我们试图优化每人的会话数。接着我们转向了并行会话，不再只是运行一个会话，而是考虑同时跑两到三个，但我是以同样的方式在运行它们，对吧？这两种模式的问题都在于，如果你让自己成为反馈循环，你现在就成了瓶颈。对吧？这些 Agent 的变快速度远早于你能够跟上的速度。所以前两个阶段我直接用一张图表来说明，我们达到了每人两到三个会话，但我们基本见顶了。

英文原文

If I think about um sort of how did we evolve with our development cycle? We started if if I go back to when our CLI product initially launched, it was one session, one person. So I'm sure many of us experienced this when these new agents came out and I was typing and it would then work and then it would stop working and then I would type again, right? We just go back and forth and then from that so that was one session, right? I said we trying to optimize sessions per person. Then we moved to parallel sessions where instead of just running one session, I'm looking at, you know, maybe I'm running two or three, but I'm running them in the same manner, right? And the problem with both of these is that if you make yourself the feedback loop, you are now the bottleneck. Right? These agents got faster long before you're going to be able to keep up. And so these first two sessions, I'm just going to kill in one chart here where it's, you know, we got to two or three sessions per person, but we sort of topped out.

现在，第三阶段引入了记忆和定时任务调度。有了记忆，一个新会话现在突然就懂得了你的规范，对吧？所以——

英文原文

Now, stage three introduced memory and schedules. And with a memory, a new session now all now all of a sudden knows your conventions, right? So

隐式记忆。接着是定时调度，有了定时调度，你就不需要自己去启动所有的工作了。事情可以按计划定时发生。我们在这里学到的一点是，只有在你的机器处于开机状态时，定时调度才会起作用。在之前的阶段，我们的大多数工程师都是在本地运行，对吧？我相信今天在座的许多人仍然是这样做的。但是迁移到远程后，你就不再需要担心“它会不会运行”了，因为我有一台一直在线的机器。这是我们在 AWS 非常擅长的事情。所以显然我们是在远程运行的。

英文原文

implicit memories. And then schedules make it so you don't have to kick all of the work off on your own. Things can happen on a schedule. The thing we learned here was that schedules only help if your machine is on. So in the previous phases, most of our engineers were running locally, right? As I'm sure many of you do still today. But moving to remote ends up making it so you no longer have to worry about is it going to run because I have an always on machine. Something we're very good at at AWS. So obviously we are running remote.

嗯，但接下来发生的情况是，这让我们能够……可以说是增加到了 10 到 20 个会话，对吧，然后我们又陷入了停滞。仅靠记忆和定时调度，我们无法打破那个瓶颈，因为我们所做的大部分工作，呃，仍然是在充当会话之间的粘合剂，对吧。

英文原文

Um but what happened then is this allowed us to sort of I'll say build up to between 10 and 20 sessions right and then we sort of plateaued again we were unable to sort of break that barrier with just memory and schedules because much of what we were doing uh was still being the glue between the sessions right

然后如果我们要……我们的下一个阶段是流水线（pipeline），但并不是大家可能想到的那种 CI/CD 流水线，而是更像一条装配线（assembly line）。嗯，其实我之前……我上一段职业生涯是在航空航天领域工作，所以接触过很多装配线。而装配线最难处理的就是非标准化工作，对吧？所以如果流经系统的每一件东西都完全相同，装配线就运作得非常好。一旦遇到非标准化工作，事情一下子就变得棘手了。我相信大家也都知道这一点，软件开发并不会每次都走完全相同的路径。

英文原文

and then if we our next phase was a pipeline but not a CI/CD pipeline like you might be thinking but instead more like an assembly line um I actually have uh my previous life I worked in aerospace so a lot of assembly lines And the hard thing with assembly lines is non-standard work, right? So if everything is exactly the same as it flows through the system, an assembly line works great. As soon as you get non-standard work, all of a sudden it becomes difficult. And I'm sure all of you know this, but software doesn't follow the exact same path every time.

于是我们发现自己的团队当时在做的事情就是这个——抱歉，这把我们带到了大概 30、20、30、40 个会话。但接下来的情况是，我们的团队当时是在试图疏通装配线或者说流水线，对吧？所以，随着问题的出现，如果我们本以为是个小问题，结果演变成了一个大问题，有时候装配线就会卡住，就必须有人介入去疏通那个东西，也许是去重启它，也许是重新规划它。嗯，所以这……这能起到的作用是有限的，因为我们又一次陷在智能体的运作当中了。相比于智能体想要工作的程度，反而是我们在更多地阻碍智能体。

英文原文

And so what we found our own teams doing then was this. Sorry, this got us to maybe like 30 20 30 40 sessions. But what happened was is our team was then trying to unblock the assembly line or the pipeline, right? So, as things came up, if we thought something was a small issue and then it turned into a big issue, sometimes the assembly line would be stuck, somebody would have to come in and unblock that thing, maybe you restart it, maybe you replan it. Um, and so this this is only going to carry us so far because again, we are in the midst of the agent. We're blocking the agent more than the agent wants to work.

Ultra 模式与多智能体编排

那么针对这一点，我们做了什么呢？我们推出了一个叫做 Ultra 模式（ultra mode）的东西。嗯，先说明一下，Ultra 模式目前……如果你去试用并寻找它的话，它目前仍处于开发者预览阶段，所以它位于功能开关（feature flag）之后。但由于它是开源的，代码都在那里。如果你想去试试，你可以体验一下。呃，就像我开头所说的，如果我们自己不喜欢它，我们就不会发布它。所以我们目前仍在对它进行最后的完善。

英文原文

And so what did we do for that? Well, we launched a thing called ultra mode. Um, ultra mode is still, just to be clear, if you go try it and you look for it, it's still in a developer uh, preview, so it's behind a feature flag, but because it's open source, the code is all there. If you want to go try it, you can you can try it out. Uh, as I said up front, if we don't love it, we don't ship it. So, we're still sort of finishing putting the finishing touches on this one.

不过我还是想讲一讲它，因为它改变了你的角色——如果我们回顾刚才这一块中间部分，嗯，由我们来充当推动整个工作的人，那些可能就是我们之前在做的事情。其中最大的一项就是盯住问题，然后在问题出现时试图去修复装配线。呃，也许你在评判一个结果，或者你在决定下一步该做什么。Ultra 模式的作用就是替你承担这项工作，对吧？也就是你在你的各个智能体之上再放一个智能体，由它来负责这些事：根据你要达成的任何目标来定义目标、启动任务、监视问题、解决卡点。而这在今天让我们达到了 50 多个会话。

英文原文

But I wanted to talk about it anyway because it takes you from being what we did was if you look at this uh, middle part here, um, where we are the person driving the effort, these are the things that we might have been doing. So watching for problems was one of the biggest ones and then trying to fix the assembly line when the problem came up. Uh maybe you're judging a result or you're deciding what to do next. What Ultra Mode does is it does that job for you, right? So you put an agent on top of your agents that then is responsible for doing that, defining the goal based on whatever you're trying to accomplish, starting things, watching for problems, unsticking things. And this got this is getting us today to 50 plus sessions.

所以如果回想一下我们当初是从单个人和一个会话开始的，现在如果单个开发者就能跑 50 多个会话，这就是为什么能在之前给你们看的 PR 图表中获得那样的体量。而且我们已经能看到即将到来的下一堵墙了。嗯，编排器与其他编排器协同工作，随后围绕你希望它成为什么样的治理机制以及护栏，就变得愈发重要了。对吧？

英文原文

And so if you think about where we started with a single session and a single person, now if a single developer is doing 50 plus sessions, that's how you get that sort of the volume in the PR chart I showed you earlier. And we can already see the next wall that's coming. Um the orchestrators working with other orchestrators and then the governance around what you want that to be and the guard rails become even more important. Right?

所以再次说明，这就是我们获得那种体量的方式。那是我们经历过的五个阶段。嗯，对我们来说 Ultra 模式彻底改变了局面，装配线也是一样。呃，如果大家觉得这些想法对自己有用，我鼓励你们去尝试一下。不过我接下来想谈的是，我们从中……我们从中领悟到了什么？究竟是什么在最大程度上阻碍了我们快速交付？

英文原文

So again, that's how we got the volume. Those were the five phases we went through. Um, ultra mode has been a game changer for us as was the assembly line and and uh I invite you to try those ideas if you think they might work for you. The next thing I wanted to talk about though was how do we what did we learn from this? Like what are the biggest things that prevented us from shipping quickly?

交付经验：全面测试与审查机制

第一条是测试一切。这并不是……这并不是什么新教训，对吧？软件行业的每个人都明白这一点。我们本就应该这么做。嗯，但我依然太频繁地看到人们……嗯，低估了这所带来的价值。而这一点将与接下来的另一项关于审查的内容结合在一起。比方说，如果去想一想，你发布某样东西的信心究竟来自哪里？我认为很多人依然觉得：“嘿，我之所以有信心，是因为我看过了所有的代码，我确切知道那段代码在做什么。”再说一次，如果你那样做，也许那是一件好事。但是我们获得信心的地方——亚马逊一直以来都是高度测试驱动的，嗯——是来自测试，因为你也许无法每次都去逐一查看代码，但你肯定能够每次都以完全相同的方式去测试某样东西，并检验其结果，对吧？从而得出结果。

英文原文

The first one is test everything. And now this is no this is not a new lesson, right? Everybody in software knows this. We should be doing this. Um, but I still see people too often um, discount the value this is providing. And this is going to build in with the next one behind this around reviews. Like if you think about where do you get confidence to ship something? I think many people still feel like, hey, I get confidence because I looked at all of the code and I know exactly what that code does. Again, if you do that, like maybe that's a good thing. But where we have gotten our confidence, and Amazon's always been very test driven, um, is from the tests because you might not be able to look at something every time, but you can certainly test something every time in the exact same way and test the result, right? And come out with the result.

所以对我们来说，质量归根结底变成了一个测试问题。我们目前把相当大一部分时间——大约 20%——专门花在测试上。

英文原文

And so for us again, quality really becomes a testing problem. And we're spending a significant portion of our time around 20% just on testing today.

我们学到的另一件大事——也是一件更大的事，同时也是一个有争议的话题——就是审查。那么，大家快速举个手。有多少人曾用智能体做过审查，本以为只会有一两条评审意见，结果回过头一看却心想：“天呐，这代码写得太烂了，对吧？到处都是评审意见。”我们也遇到过同样的问题。

英文原文

The other big thing we learned, and this is a bigger one and also a contentious topic, is is reviews. So, quick show of hands. How many people have done a review with an agent thinking that there's going to be like one or two comments and then you come back and you're like, "Oh my god, like this is terrible code, right? There's comments everywhere." We had the same problem.

所以我们最初写了一个审查智能体，它会审查所有的内容，而我们的目标是修改每一条评审意见。好吧，大家……我听到笑声了。如你们所料，你提交了修改。你提交了一个 PR，收到了关于该 PR 的反馈意见。修复之后，你再次提交。回头一看，现在又冒出了三个之前测都没……之前甚至都没提过的新问题，并且在已经修复的地方又冒出了四个问题。你再次提交，这个循环就这么不断重复。因此，我们现在在审查方面所做的，是对审查红线（red reviews）进行分类或分流，分为阻塞性问题和非阻塞性问题。比如破坏构建的情况，嗯，如果有明显导致构建失败的问题，那就是阻塞性问题，在合并这个 PR 之前我们必须先去解决它。改个变量名，没什么大不了的，我们稍后再处理。所以对于这类问题我们就放行通过了。

英文原文

So, we originally wrote a review agent that would review everything and our goal was to fix every single comment. Well, as you guys I heard the laughter. So, as you expect, you you submit changes. So, you submit a PR, you get the feedback on that PR. You come back to you submit it again after you fixed it. You come back and now it's three new things that weren't even test weren't even told about before and it's four things on the things you already fixed. You submit it again and this cycle just continues to repeat. So, what we've done with reviews is now we categorize the red reviews or triage them into blocking and non-blocking issues. So, for example, something that breaks the build, um, if there's something obviously that breaks the build, that's a blocking thing and we need to go fix that before we merge this PR. Changing a variable name, not a big deal. We'll take care of that later. So, we let those things slide through.

我们在审查方面做的另一件事是，我们[清嗓子]希望智能体能够获得快速的反馈。正如大家在这里可以看到的过程——我稍后会带大家了解——这并不是那么快。呃，所以我们设有一个快速关卡，它会非常迅速地进行复核，确保比如在进入正式审查之前文件都放对了目录。

英文原文

The other thing that we've done with reviews is we [clears throat] want to have fast fast feedback for an agent. And as you can see by the process here that I'll walk you through in a second, this is not that fast. Uh, and so we have a fast gate that really double checks things really quickly to make sure that like things are in the right folders before we move on to our reviews.

实际上我们运行着六个审查器：两个代码审查器，分别运行在不同模型和不同模型家族上。嗯，目的是……那里的意图是切实发现……确保不会出现模型盲区，对吧？以免模型由于未在某些内容上训练过而导致我们没能捕获问题。这里就是我们使用 Anthropic 和 OpenAI 模型或者开源权重模型来作为那些杠杆的地方。

英文原文

We have six reviewers actually that run two code reviewers both in different models with different model families. Um, intended and the intent there is to really find to not make sure you don't have a model gap, right? So that the model wasn't trained on something that we don't catch. This is where we use the um anthropic and open AAI models or an openw weight model to be those uh levers.

我们还运行一个设计审查，询问：这个东西的形式是否正确？也就是说，我正在构建的内容——比如如果我在此 PR 中添加另一个缓存，我现有的缓存是否已经能解决这个问题？我们有一个第一性原理审查，询问：这次变更是否解决了真正的问题？嗯，我们同样还有一个 UX 审查，如果你在做一个……比方说你在新建一条错误提示信息，我们会让智能体根据已经发布的内容，来判断人类是否能够根据该错误信息知道该怎么做。接着，我们还在持续收紧安全态势。

英文原文

We run a design review that asks, does this thing have the right shape? Meaning, is what I'm building, like if I'm adding another cache in this PR, can the cache I already have take care of this issue or not? We have a first principles review that says, is this change addressing the real problem? Um, we have a UX review as well that if you're making a, let's say you're creating a new air message, we have an agent decide whether or not that air message, the human will know what to do with that air message based on what you we've published. And then we're consistently tightening our security posture.

现在，这些审查器每一个都拥有一票，非“是”即“否”，要么是红灯要么是绿灯。如果全部亮绿灯，它就进入合并。如果不是，我们就会进入仓库里的 prepare PR skill。如果大家想看，可以去查看这个 skill。然后那个 prepare PR skill 就会随着流程推进尝试去解决这些问题。

英文原文

Now each one of these reviewers gets one vote. It's either yes or no. It's red or it's green. If everything's green, it goes on to merge. If not, we go to a prepare PR skill that's in the repo. So you can check the skill out if you want. And that prepare PR skill then tries to resolve the issues as we go.

第三个教训——抱歉，关于审查还有一件事。正如我所说，我们很久之前就不再人工审查代码了

英文原文

The third lesson and sorry on the reviews. One more thing. Like I said, we've stopped reviewing code a long

……很久以前的事了。我们已经往前走了，而我们所负责的并不是代码本身。看看这个能不能翻回去。确实是这样——我们所负责的是审查记录。它是否按照我们的预期遵循了流程，以及我们是否得到了想要的结果？如果没有，我们就改进审查流程。

英文原文

time ago. We have moved on and what we're responsible for is not the code itself. See if this actually goes back. does what we were responsible for is the review record. Did it follow the process as we expected and did we get the results we wanted? If not, we fixed the review process.

另外还有两点。那么，呃，第一点是过度工程（overengineering）的问题。因此，我们有一个专门的规则来教导我们的智能体学会做减法，而不仅仅是做加法。所以，我们尽量在推进过程中从流程中移除代码，以防止那种过度工程。我们在路线图上也采用了同样的理念；在如今，说“好”正变得非常廉价，或者说已经变得非常廉价了，对吧？现在借助写代码，我们很容易就会说“好的，我会做那个，我把这些全做了”。而过去，我们必须对我们要构建什么非常挑剔。总是说“好”的问题在于，你可能会失去你的产品形态。我们已经在 Kira Kriet 中看到了这一点，它的发展如此迅速、节奏如此之快，以至于你不想让你的用户陷入杂音之中。所以再次重申，在路线图上使用同样的原则就是我们的做法。关于经验教训的最后一点是，合并 PR 并不意味着你就大功告成了，对吧？后面还有其他阶段。所以对我们来说，合并并不是我们所做的最后一项检查。相反，我们会运行一个发布门禁（release gate），智能体会下载代码仓库。它会安装它、运行它，然后检查核心流程，以确保整个系统在我们要进行发布之前仍然处于良好状态。接着我们也会一直进行安全检查。所以我们有，呃，每小时、每天以及常态化的——每小时对照我们的安全准则进行检查，每天对照应用安全（appsec）审查进行检查，然后还有一个常态化的渗透测试智能体。所以再次强调，合并并不是流程的终点。你必须把所有事情都压实。

英文原文

Two other things. So, um, one, the overengineering problem. So, we have a specific role to teach our agents to subtract, not just add. And so, we try to remove code from the process as we go to prevent that overengineering. And we use that same philosophy on our road map as well, where saying yes is becoming very cheap or has become very cheap, right? And it's very easy for us now with code to say like yes, I'll do that and I'll do all of these things. Whereas we used to have to be very particular about what we were going to build. The problem with saying yes all the time is that you might lose your product shape. And we're already seeing this in in Kira Kriet that it's moving so fast and so quickly that you don't want your users to get caught in the noise. And so again, use the same principle on the road map is what we do. And then last thing on lessons is you're not done once you merge a PR, right? There's still other phases. And so for us merging is not the last check that we do. Instead we run a release gate where the agent will download the repo. It installs it, runs it and then checks the core flows to make sure that the overall system is still in good shape before we would do a release. And then we do security all the time as well. So we have uh every hour every day and always on every hour against our security guidance every day against an appseac review and then an always on pentest agent. So again merging is not the end of the process. You have to crunch everything down.

所以，我要回到我最初开始讲的地方。呃，你是否已经从你的智能体中获取了全部潜力？我们很多人都觉得我们目前还没有做到。而我今天在这里要提出、或者说我尝试提出的观点是：智能体在很久以前就已经足够快来处理这项工作了。而且它只会变得越来越好、越来越好。因此，我们一直在做的就是挑战我们的工作方式，并弄清楚我们究竟在哪里阻碍了智能体尽可能快地开展工作。我今天已经给大家举了无数个例子。再次说明，我在这里只提其中的几个。首先是我们的注意力跨度（attention span），对吧？当我充当反馈循环时，我就无法再进行交付了，因为我成了那个反馈循环。当我不在线时，智能体就停止工作了。笔记本电脑屏幕一盖上，你突然之间就失去了连接。迁移到远程可以解决这个问题，对吧？有着固定形态的固定流水线在事情稍微偏离轨道时无法应对。再次强调，你该如何解决这个问题？你——我们通过一个编排器解决了它。超级远程。还有无休止的审查——我想我讲这部分的深度可能已经超过了你们感兴趣的程度，但是呃，无休止的审查周期，以及那个只会做加法的循环，对吧？那个只往里面添加东西而不尝试做减法的循环。

英文原文

So, I'm going to come back to where I started. Um, are you getting everything from your agent? And many of us don't feel like we are yet. And what I'm going to offer here today or what I've tried to offer is the agent was fast enough to work on this a long time ago. And it's only getting better and better and better. And so, what what we have been doing is challenging how we work and figuring out like where are we preventing the agent from working as quickly as possible. And I've given you numerous examples today. Again, I'll just call out a couple of them here. So, it started with our attention span, right? When I was the feedback loop, I could no longer ship because I was the feedback loop. When I wasn't there, the agent stopped working. The laptop lid closing, all of a sudden, you lose your connection. Moving to remote can solve that, right? The fixed pipeline with a fixed shape that can't respond when things go slightly off path. Again, how do you fix that? You we fixed it with an orchestrator. Ultra remote. the endless reviews that I think I've gone into probably more depth than you were interested in, but uh the endless review cycle and then the loop that only adds, right? The loop that only adds things to it and doesn't try to subtract.

最后我想留给大家的是，我也想明确一点，这将会是一个持续的调整，对吧？这并不是——这可能是在今天对我们有效的方式。我向你们保证，再过一个月，我们又会发现其他有效的方式，并且我们将继续大幅演进。但正如我在开头所说的，我们终于能够达到的交付速度，对于我们来说，我感觉终于在兑现 AI 的承诺了，但我仍然看到其他团队之间存在着差距。

英文原文

And so I'll leave you with I also want to be clear that like this is going to be a continual uh adjustment, right? This is not like this may be what works for us today. I promise you in another month we're going to find something else that works and we're going to continue to evolve significantly. But like I said in the beginning, the pace that we're finally able to deliver at, I feel like for us is that finally we're we're meeting that promise of AI and and but I still see the disparity among other teams.

所以正如我所说，我最后想留给大家这些内容。如果你有兴趣尝试 Hero Crew 并构建你自己的软件工厂，请扫描这个二维码。这是一个工作坊。呃，我们在 Kuro 上提供了相当慷慨的免费试用。所以你可以去注册免费试用，去安装 Kuro Crew，呃，然后你就可以顺利运行起来了。呃，如果你不想——如果你想尝试，就像我说的，如果你想尝试 Creoare，你也可以那样做。同样的免费试用适用于所有产品。因此，IDE、CLI、Web 客户端、移动端客户端，呃，用那个免费试用把它们都试一遍吧。说到这里，我很期待看到你们大家构建出的成果。非常感谢大家抽出时间。谢谢。祝大家在接下来的会议中过得愉快。[掌声]

英文原文

And so like I said, I'm going to leave you with this. If you're interested uh in trying Hero Crew and building your own software factory, uh scan this QR code. It's a workshop. Um we have a pretty generous free trial on Kuro. So you can go sign up for a free trial, go install Kuro Crew, um and you'll be off and running. Uh, and if you don't if you want to try, like I said, if you want to try Creoare, you can do that. The same free trial works across all of the products. So, the IDE, the CLI, the web client, the mobile client, uh, try them all with that free trial. And with that, I'm excited to see what you all build. I appreciate the time. Thank you. Enjoy the rest of the conference. [applause]

Heat. Heat. Come on. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Hey, hey, hey. Heat. Heat. Hey, hey, hey.

英文原文

Heat.

Heat.

Come

on.

Heat.

Heat.

Heat.

Heat.

Heat.

Heat.

Heat.

Hey, hey, hey.

Heat.

Heat.

Hey, hey, hey.

利用多智能体探索 Claude 前沿能力

好的，非常高兴能来到这里。呃，翻页器好用，电脑也正常。呃，我是来自 Anthropic 的 Lance。我知道我们在之前的一场主题演讲中遇到了一些问题，所以我们现在准备就绪了。我将讲述两个关于充分利用 Claude 以及探索 Claude 前沿能力的故事。首先是多智能体。这是我在 fable 52 发布时写的一篇博客，我想讲述一下这里面发生的故事。

英文原文

All right, it's very nice to be here. Um, the clicker works, the computer works. Uh, I'm Lance from Anthropic. I know we've had some issues with the one of the prior keynotes, so we're set to go. I'm going to tell two stories about getting the most out of Claude and exploring Claude's frontier capabilities. So, the first is multi-agent. This is a blog I wrote with the fable 52 launch and I want to kind of tell the story of what happened here.

费马大定理是数学界较为著名的断言之一。它大约在 350 年的时间里一直未被证明，而且它其实是一个非常简单的陈述：对于任何大于 2 的正整数，方程 $a^n + b^n = c^n$ 都没有解。然而，这个看似简单的陈述已被证明极难被真正证明。在 90 年代中期，安德鲁·怀尔斯爵士确实证明了它。呃，他发表了一份证明，这花了他整整七年的工作。过程极其艰辛。

英文原文

So for math last theorem is one of the more kind of famous um claims in mathematics. It's stood and proven for around 350 years and it's actually a very simple statement just a to the n plus b to the n equals c to the n has no solution for any positive whole numbers greater than two. Now this deceptively simple kind of statement has proven to be quite difficult to actually prove. In the mid 90s, Sir Andrew Wilds uh did prove it. Uh he put out a proof and it took him seven years of work. It was extremely painstaking.

自那时起，数学界的一个重要工作和兴趣点就是将该证明形式化，或者说从基本上通过计算来检查每一步。证明本质上是一长串逻辑链条，而你实际上可以使用不同的系统来通过计算检查证明，我稍后会介绍，比如使用 Lean。

英文原文

And since that time, a major effort and interest in mathematics has been formalizing that proof or basically computationally checking every step. So proofs are kind of long chains of logic and you actually can computationally check proofs using different systems which I'll talk about for example using lean.

所以这只是一个大致说明的视觉画面——这是一个非常有趣的视频，展示了他所经历的努力，要建立这个证明是一个极其艰辛的过程。

英文原文

So this is just kind of a visual kind of explaining the this is a very interesting video showing the effort that he went through was extremely painstaking process to you know establish this proof

而在 9 月，我们发布了我们使用 Claude 将其形式化的成果。这算是我们在这方面分享的第一个非常重要的成果，我们使用了 Lean 来完成它。Lean 允许你通过计算来验证证明中的每一步。不过，做这件事情极其艰辛。我们一开始使用了一个十几名智能体组成的团队，这里大致展示了那个团队的架构图。在最初阶段，那个团队工作得并不是很有效率。我们必须给它提供一个叫做 prove to me 的东西。这是一种用于编排数学证明的平台，它极大地帮助了智能体管理项目的状态。

英文原文

and in September we published that we used claw to formalize it. This was like kind of the first very important result that we kind of shared along these lines and we used lean to do it. So lean allows you to computationally verify every step in a proof. Now doing this though is extremely painstaking. We started with a team of down a dozen agents and this is kind of just showing a schematic of that team. Very early on that team didn't work very effectively. We had to give it something called prove to me. Now this is a kind of a platform for orchestrating mathematics proofs and it helped the agents significantly manage the state of the project.

你可以把 Lean 看作一个测试载具（harness）。基本上，它为智能体提供了一个可供查询的图。所以它本质上是围绕这个问题陈述的大量脚手架。你可以把它看作一个高度特定于领域的测试载具。好的。现在，这些全都是在为我们实际做的事情做铺垫。我们汇报过这是在 9 月份的事，也就是形式化。现在，对于 F——对于 52，我想做的是展示我们能否拥有一个规模大得多的智能体团队。也就是从大约十几名扩展到一千名，在没有领域特定测试载具、没有 prove to me、并且在解决方案方面没有人为引导的情况下来完成这项形式化工作。我们在 9 月份谈到的那份报告中，既包含了这个 prove to me 领域特定测试载具，也包含了来自研究人员的大量支持。而这一次是测试一大群 Fable 2 智能体能否完全自主地自行开发出这种形式化。

英文原文

Lean you can think about as a harness. So basically it provides the agents with a graph that they can query. So it it's basically lots of scaffolding around this problem statement. You can think of it as like a a very domain specific harness. Okay. Now this is all setting up what we actually did. We reported this was back in September. The formalization. Now what I wanted to do for f for 52 is show can we have a much larger team of agents. So going from around a dozen to a thousand do this formalization with no domain specific harness. without proved to me and with no human steering with respect to the solution. So the report we talked about in September had both this domain specific harness proved to me and a lot of support from a researcher. This is testing whether or not a large team of Fable 2 agents can autonomously develop this this formalization themselves.

这里只是让大家对这个测试载具（harness）有一个感性认识。它看起来有点复杂，但实际上相当简单。这一千名智能体中的每一个都有其自带受限网络访问权限的独立沙箱。其中大部分都是针对安全和防护的。因此我们有一千名智能体。每个都有它们自己的沙箱。它们可以连接到两样不同的东西：Git 和一个 SQL 数据库。仅此而已。那就是唯一的——那些就是我们赋予这个团队的核心原语。并且我们有实时监控在查看每一个工具调用，以防逃逸等情况。我们还有离线监控，每 10 分钟查看一次所有智能体的运行情况。这大致就是整体的设置：千名智能体团队，非常简单的原语，只有 Git 和一个数据库。

英文原文

So this is just giving you a sense for the harness. It looks kind of complicated but it's actually pretty simple. Each of the thousand agents has its own sandbox with restricted network access. Most of this is directed on safety and security. So we have a thousand agents. Each have their own sandbox. They can connect to two different things. Git and a SQL database. That's it. That's the only those are the core prims that we give this team. And we have real-time monitoring looking at every tool call for things like escapes. We also have offline monitoring looking at runs of all the agents every 10 minutes. This is kind of the setup. thousand agent team very simple primitives just get in and a database

提示词（prompting）相当简单，它确立了一名主导智能体（lead agent）、9999 名助手智能体，赋予它们对 Git 和数据库的感知，以及……

英文原文

the prompting is pretty simple it establishes one lead agent 9999 helper agents it gives them awareness of git and the database and

问题陈述，其实就是这样。我们想看看这个团队能否接着完成这个形式化工作。

英文原文

the problem statement that's really it we want to see if this team could then do this formalization

有趣的是，这大体就是该团队所做事情的过程。Lead（主导智能体）在非常早期，也就是运行开始五分钟后，基本上就创建了这些数据库表。

英文原文

what's interesting is this is kind of the story of what the team did the lead very early on five minutes into the run basically created these database tables.

所以智能体现在已经足够先进，以至于它们实际上可以构建自己解决棘手问题所需的原语。

英文原文

So agents now are sufficiently advanced that they can actually build primitives that they need to solve hard problems.

我们并没有告诉它们去建这些表。这些表实际上就是智能体自己建出来的。

英文原文

We didn't tell them to build these tables. These are the tables that the agent actually just kind of built.

一个可以让 Lead 发布通告、给 worker 分配角色的表；大家，你知道的，智能体可以在里面发 note 的表；一个 subtask 表；一个用来指示事情是否已经 landed（合入/落地）的表。所以 Lead 就直接创建了这些表来协调编排这个团队。这是一个可视化图表，展示了智能体在这次运行过程中如何向这些表进行写入和读取。

英文原文

A table where the lead can announce things, assign roles to workers, everyone, you know, agents can post notes, a subtask table, a table that indicates whether or not things are landed. So the lead just creates these tables to orchestrate this team. And this is a visualization showing how the agents wrote to and read from these tables over the course of this run.

现在，顶部的红色表示对表的写入。绿色表示读取。真正有趣的是，你可以看到 Lead 偶尔会向 announce 表写入。这大致是发布关于这个问题的重大全局性公告。

英文原文

Now red on the top is showing writes to the tables. Green is showing reads. What's really interesting is you can see occasionally the lead writing to the announce table. This kind of announces major global like kind of um announcements about the problem.

嗯，它还会随着定理证明的推进，给 worker，即这些 helper 分配该定理中的不同命题。Notes 是它们沟通并分享进展情况的地方。

英文原文

Um it also assigns workers these helpers different statements in this theorem as they proceed. Notes are where they communicate and share kind of progress.

Landed 是它们实际写出作为证明一部分的 Lean 代码的地方。因此，所有智能体完全自主地、在没有具体 prompt 指导的情况下，以这种方式使用数据库来协调它们在这个极其困难的长程任务上的工作。这运行了大约 30 个小时。

英文原文

Landed is where they kind of actually write the lean code uh as part of the proof. And so the agents all autonomously without specific prompting use the database in this way to coordinate their work on this extremely hard long horizon task. This ran for around 30 hours.

对于这个证明中特别具有挑战性的部分，它们实际上开发了自己的抽象。这其实是关于这个 lran's tunnel 的任务树。这是证明中较难的命题之一。

英文原文

And for particularly challenging parts of this proof, they actually developed their own abstractions. This is actually a task tree for this lran's tunnel. This is one of the harder statements in the proof.

而这里实际上展示了这棵树内任务的不同依赖关系。这里实际上展示了随着时间的推移，进展是如何从进行中变为红色的完成状态的。

英文原文

And this is actually showing the different dependencies of the tasks within this tree. And this is actually showing over time how you know the progress from like in progress to done in red.

所以所有这些的重点，其实主要就是想强调智能体可以在不需要太多脚手架（scaffolding）的情况下解决非常长程的问题。这里展示的差不多是在本次运行过程中，智能体在这些不同任务间的总体分配情况。

英文原文

So the point of all this is really just to to kind of highlight agents can solve very long horizon problems without much scaffolding. This is showing kind of the overall kind of um allocation of the agents across these different tasks over the course of this run.

嗯，这大致展示了随着进展推进以及随着这些命题——这个证明中有 60 个命题——全部得到解决，Lead 和智能体自身是如何重新分配自己并重新指派给问题的不同部分的。因此，智能体使用简单的原语、git 和数据库，就能够在这个非常棘手的问题上自行协调，随着时间的推移逐步解决它。这确实是关键要点，你可以在这里看到时间的推移。

英文原文

And um this kind of shows how the lead and the agents themselves kind of reallocate themselves and kind of reassign themselves to different parts of the problem as it progresses and as these statements there's 60 statements in this proof are all solved. So the agents using simple primitives git and the database can coordinate themselves across this very hard problem to solve it progressively over time. That's really the key point and you can see time progressing here.

这是来自我们的一次运行。这是智能体留下的其中一条 note，基本上写着：即刻起原点全面冻结（total freeze of the origin effective now）。你们全部 999 位干得漂亮。

英文原文

this from one of our runs. This is one of the notes that the agents left basically saying total freeze of the origin effective now. Magnificent work all 999 of you.

嗯，所以基本上这件非常有趣的事情是，我们得以展示这项形式化工作，而在 9 月份时，人们原本预计这需要耗费许多许多年。我们最初展示了可以用 12 个，用大约一打智能体加上大量的 scaffolding 来完成。这套 prove-to-me harness 以及来自研究人员的大量推敲介入与支持。

英文原文

U so so basically what is really interesting about this is we were able to show this formalization which was expected to take many many years back in September. We showed initially we could do it with 12 with like a dozen agents with a lot of scaffolding. This proved to me harness and a lot of poking and and and support from a researcher.

这个新结果展示了，嗯，我们在安全和监控方面提供了很多结构支持，但在这个具体案例中，智能体团队能够使用非常简单的原语完成这项任务，而无需针对特定领域的 harness。而且智能体在自我编排以及知道如何针对极具挑战性的长程任务使用这些简单原语进行协调方面，正变得越来越出色。这确实是这项工作中非常有趣的核心观察。

英文原文

This new result is showing um we provide a lot of structure in terms of safety and monitoring but the agent team was able to accomplish this task with very simple primitives without a domain specific harness in this particular case. Um and agents are getting increasingly good at orchestrating themselves and knowing how to coordinate using these simple primitives for very challenging long horizon tasks. That's really the key kind of observation which is very interesting from this work.

智能体记忆

现在我想转个方向，聊聊记忆（memory），这与我之前在多智能体方面所讲的完全不同，但它突出了我们看到 Claude 获得显著能力的另一个领域。之前我们差不多谈到了 Claude 编排大型团队以解决非常长程问题的能力。记忆则是一项不同的能力。它表明 Claude 越来越有能力组织这种上下文，以帮助解决长程问题。

英文原文

Now I want to shift gears and talk about memory which is uh entirely different when I talked about in terms of multi-aent but it's highlighting at another area where we're seeing claude gain significant capability. So previously we talked kind of about cloud's ability to orchestrate large teams to solve very long horizon problems. Memory is is a different capability. It's showing that claude is increasingly capable to organize this context to help solve long horizon problems.

我前不久就此发表了一篇博客，而这实际上是博客的开篇第一句话。我认为这实际上概括了该博客的想法，而且作为关于记忆的核心想法之一，我觉得非常有趣，这是来自哲学家克尔凯郭尔（Kierkegaard）：

英文原文

So I published an blog on this fairly recently and this is actually the lead line of the blog. I think this actually encapsulates the idea of the blog and like one of the central ideas around memory I think is very interesting and this is from the philosopher Kiergard.

生活只有往后看才能被理解，但必须往前过。这是智能体记忆的一个根本悖论，因为智能体写入记忆通常是在它们执行任务的过程中，遇到了奇怪或有趣的事情，就会写入记忆。但是记忆需要成为智能体知识的一张连贯地图，以便智能体可以回溯和反思。因此，当它们向记忆写入时，它们的写入是带有某种短视性的。

英文原文

So life can only be understood backwards but it must be lived forwards. This is a fundamental paradox of agent memory because agents write to memory typically as they go about a task they encounter something weird or interesting they write to memory. But memory needs to be a coherent map of an agent's knowledge that agents can look back on and reflect on. So when they're writing to memory, they're writing kind of myopically.

它们在一个 session 内贪婪地审视并想着：“啊，这对眼下的任务似乎很重要。”但是记忆需要是全局连贯且不冗余的。所以这是记忆的一个非常棘手的悖论。坦率地说，我们人类也面临着同样的挑战。我们在日常生活中活动，写入短期记忆。据信海马体大致就是负责编排此过程的地方。到了夜间，据信海马体会向存储长期知识的大脑新皮层进行回放。

英文原文

They're looking greedily within a session saying, "Ah, this seems important for this task at hand." But memory needs to be something that's globally coherent and not redundant. So this is like a very tricky paradox of memory. And frankly, we as humans have the same kind of challenge. We go about our day, we write to short-term memories. It's believed that the hippocampus kind of is the place that that's orchestrated. At night, it's believed that the hippocampus replays to the neoortexes where long-term knowledge is stored.

呃，因此人们认为，人类的这种做梦过程实际上有助于将我们日常经历的碎片或最重要的方面巩固整合为长期知识。因此，在日常记忆和某种长期知识之间存在这种解耦。我们发现智能体也能从同样的解耦中获益。这确实是该文章的核心要点。

英文原文

Uh, so it's believed that actually this dreaming process for humans help consolidate the fragments or like the most important aspects of our daily experience into long-term knowledge. So there's this decomposition between daily memory and kind of long-term knowledge. And we found agents can benefit from the same decomposition. That's really the key point of the post.

现在，这里有一些说明情况的结果。这实际上是一个名为 CLB bench 的开源持续学习基准测试，它表明随着你从 Sonnet 一路往上到像 Open 5 这样的不同模型，它们在使用记忆方面变得越来越出色。

英文原文

Now, here's some results kind of telling the story. So this is actually an open source continue learning benchmark called CLB bench and this is showing that uh different models as you go from like sonnet up to like open 5 get increasingly good at using memory.

这项任务的设置非常简单。它要求 Claude 去回答关于一个数据库的 40 个问题。在每个问题之间都是一个全新的 session，Claude 可以在每个 session 之间写入笔记，比如一个记忆目录。所以它差不多像是一组连续的问题，Claude 可以写入记忆来帮助应对后续的问题，对吧？

英文原文

Now this task is a very simple setup. It asks claude to ask to solve 40 questions about a database. In between each question it's a fresh session and claude can write to notes like a memory directory between each session. So it's kind like a set of sequential questions and Claude can write memories to help with future questions, right?

那么实际发生的情况是，深色柱状图表示为你给 Claude 配备了记忆存储时的情况，而阴影线则表示没有记忆的情况。好的。现在有趣的是，对于 Sonnet 46，赋予 Sonnet 记忆——你可以从深蓝色柱中看到——收效甚微。基本上无法真正使用记忆。我们查看了结果并发现，Sonnet 并不能可靠地知道何时该写入记忆，而且也不会很自然地从记忆中读取。

英文原文

So what happens is the the dark bars are showing when you give Claude the memory store and then the the hash is like without memory. Okay. Now what's interesting is with Sonnet 46, giving Sonnet memory, you can see in the dark blue has very little benefit. Can't really use memory. and we looked at the results and we found sonnet doesn't um doesn't reliably know when to write to memory and also doesn't really uh naturally read from memory.

好的，同样值得注意的是，这些实验是在没有特定记忆 prompt 的情况下完成的。它只是告诉智能体该目录可用，如果想用的话可以使用记忆。其实仅此而已。因此 Sonnet 46 原生上无法有效地使用记忆。Sonnet 5 确实表现更好，Opus 也更好，所以基本上模型在相同的简单指令下，使用记忆的能力变得越来越强。

英文原文

Okay, it's also worth noting that these experiments are done with no specific memory prompting. It just tells the agent the directory is available use memory if you want. That's really it. So sonnet 46 doesn't natively use memory effectively. Sonet 5 does does indeed better opus and so and so basically the models get better and better with the same simple instruction at using memory.

而这里真正的核心观点是，更强的模型所表现出的是，它们内在地知道何时向记忆写入，并且它们内在也大致知道何时从记忆中读取以检索信息。它们的记忆也更好。它们就像是高度经过验证的陈述，或者是关于数据库状态或接下来可能发生什么情况的预测，相比之下，较早的模型往往会写下质量较低的记忆。

英文原文

And really the key idea here is that what happens is with stronger models they know intrinsically when to write to memory and they kind of know intrinsically uh when to read from memory to retrieve information. Their memories are also better. They're like they're highly verified statements or predictions about kind of the state of the database or what may happen next versus earlier models tend to write um lower quality memories.

我将向大家展示另一个游戏中非常有趣的一个例子。这是来自宝可梦（Pokemon）。这里向大家展示的是在宝可梦游戏中，Claude 37 的记忆目录与 Fable 5 的对比。嗯，这里的关键点是：每个方框的大小表示在每个目录中记忆文件的数量。好的。

英文原文

And I'll show you a very fun example of this from a different game. This is from Pokemon. So this is showing you Claude 37's memory directory from Pokemon versus Fable 5. Um here is the key point. The size of each box indicates the um the number of memory files in each of these directories. Okay.

所以当我们让 Claude 玩宝可梦时，我们给 Claude 一个记忆目录，就像我在之前的基准测试中所做的那样，对吧？Claude 在玩游戏时可以随心所欲地往那个目录里写任何内容。真正有趣的是，Sonnet 37，我们看到它向记忆中写入了极其庞大的内容。而当你去查看这些记忆时，它是毫无选择性的。基本上就是在记下各种各样的经历。它创建了这样一个极其庞大、难以处理、简直令人作呕的记忆目录，里面充斥着对[……]而言并非真正相关或具有战略意义的内容。

英文原文

So when we have cla pay Pokemon, we give Claude a memory directory just like I did in that prior benchmark, right? Claude can write whatever it wants that directory while it plays the game. What's really interesting is Sonnet 37, we see it writes a huge amount to memory. And when you look at the memories, it's indiscriminate. It's basically writing down all sorts of experience. It creates this huge unwieldy just disgusting memory directory of things that are not really relevant or strategic for

游戏进行得远不如 Fable 5。Fable 5 的记忆目录紧凑得多。它包含的完全是战略洞见、战术知识，是那些你无法轻易从游戏状态中直接获得的东西。所以我们看到的是，更优秀的模型具有更好的显著性直觉，更清楚该往记忆中写入什么。你可以在记忆目录的大小上看到这一点反映出来。它紧凑得多，战略性强得多，而且 Fable 5 在游戏过程中确实比 Sonic 37 走得远得多。这从另一个角度印证了一个直觉：Claude 在写入记忆方面表现好得多，清楚该写什么、什么是显著的，而且在利用和读取记忆方面也是如此。因此模型在感知显著性、知晓写什么以及读取记忆方面正变得越来越有效。

英文原文

gameplay and it proceeds much worse in the game than Fable 5. Fable 5's memory directory is much tighter. It's really strategic insights, tactical knowledge, things you can't just get from game state very easily. So what we see is better models are much are have much better instincts for salience and like what to write to memory. And you can see this reflected in the size of the memory directories. It's much tighter. It's much more strategic and gameplay indeed with Fable 5 gets much further than Sonic 37. So it gives another intuition around the fact that Claude is much better at writing to memory, knowing kind of what to write in salience and what's salient and uh and also kind of using and reading from memory. So models are getting more and more u kind of effective at knowing salience, knowing what to write and also reading.

现在，这两种情况都属于我在最初框架中所说的在经历过程中写入记忆。基本上，在这两项任务中，我们都让 Claude 在玩这个游戏时写入记忆。我们让 Claude 在运行 Sealbench 基准测试时写入记忆。但是正如我提到的，记忆还有梦境这一面。这能让 Claude 对记忆进行反思，从而修正问题。我会展示一些这方面的有趣示例。

英文原文

Now, both of these represent what I would consider in my initial framing of like writing to memory during experience. So, basically in both of these tasks, we let Claude write to memory while it plays this game. We let Claude write to memory while it does the Sealbench benchmark. But, as I mentioned, there's also the dreaming side of memory. So, this is allowing Claude to reflect over memories to fix things. And I'll show some interesting examples of that.

梦境机制与记忆构建

这是关于梦境的一种构想框架。你可以把记忆理解为……嗯，存在转录记录。转录记录可以被视为记忆的基石。想象一下，一个记忆系统可以非常简单，仅仅是转录记录搜索。Claude 在执行某项任务时写入转录记录。未来的会话可以从转录记录中读取。当然，转录记录体量庞大，非常笨重。我们在博文中反复谈到这一事实：转录记录中通常包含的信息……你知道，转录记录中其实只有极少极少的一部分信息是显著且重要的。因此我们也允许 Claude 执行诸如撰写记忆笔记这样的操作。这就是我刚才谈到的内容。随着任务推进，它可以向记忆写入笔记。但梦境能让你做得更多。梦境让你能够对转录记录以及这些写下的笔记进行反思，并重新表征它们。例如，构建像 wiki 这样的记忆表征。甚至可以将某些记忆提升为注入式上下文，并预先加载到未来的每一个会话中。所以梦境是一种机制，通过它你可以处理现有的转录记录或现有的记忆笔记，并从中构建新的东西，包括表征、注入式上下文等。

英文原文

This is kind of a a framing of dreaming. And so you can think about memory in terms of um there are transcripts. Transcripts you can think about as kind of the foundation of memory. So imagine a memory system could very simply be just transcript search. Claude writes to a transcript as it goes about a task. Future session can read from the transcript. Of course transcripts are large. It's very unwieldy. We talk a lot about in the blog post the fact that transcripts often contain information that you know a very very small amount of information in transcripts is actually salient and matters. So we also allow claw to do things like writing memory notes. That's what I just talked about. As it goes, it can write notes to memory. But dreaming lets you do more. Dreaming lets you reflect over transcripts and these written notes and re re-represent them. Build memory representations like wikis as an example. Actually promote certain memories to injected context and load it preemptively into every future session. So memory is this dreaming is this mechanism where you can kind of process existing transcripts or existing memory notes and build new things from them including representations, injected contexts and more.

来看一些这方面的有趣示例，这是来自《宝可梦》游戏的实机情况。梦境是纠正记忆中错误的有效途径。正如我前面谈到的，当 Claude 执行任务时，它可能会在记忆中写下不正确的笔记。就是它犯了个错。这发生在这场特定的游戏中。嗯，Claude 错误地记下了游戏中某个地标的位置。伴随着这条错误写入的记忆，发生的情况是 Claude 陷入了一个死循环：基本上它会在游戏的这个特定节点误判自身位置，掉进这个活板门陷阱，然后基本上被送回上一层。这就是大体上的逻辑。具体细节并没有那么有趣或重要。关键思想在于，一条有毒的记忆被注入并卡在了那里。梦境能够化解这一困境。这就是为什么梦境在对记忆进行反思和修正方面极其强大的直观原因。

英文原文

So some fun examples of this, this is from Pokemon gameplay. Dreaming is an effective way to correct mistakes in memory. So as I talked about when Claude's going about a task, it can write incorrect notes to memory. Just it makes a mistake. This happened this is in in this particular game. um Claude wrote incorrectly wrote the location uh of a particular landmark in the game. Now what happened is with this incorrectly written memory, Claude got trapped in a loop where basically it would misloize itself at this particular point in the game, fall down this trap door and it would basically send it back to the prior level. That's like the highle intuition. The details aren't that interesting or important. The key idea is a poison memory was injected and it's stuck there. Dreaming can rescue that. That's really the intuition on why dreaming is extremely powerful for reflecting over memories and correcting them.

这里展示了游戏的一些运行轨迹，其中橙色代表记忆与梦境。向上代表好的态势。基本上，向上代表游戏进展，向下代表某种游戏倒退。有趣的是，对于原始记忆，嗯，基本上在没有梦境的情况下，我们可以看到它崩溃了——植入了一条错误的记忆，而在到达那个特定节点时，Claude 在游戏中继续倒退或跌落；而有了梦境，它化解了那个错误，Claude 推进到了下一层。这确实是关键的观察结论，而且这是一个非常通用的原理：Claude 可以利用梦境来修正记忆中原本卡在那里的错误。这是一种能够纠正错误的重要途径。

英文原文

So this is showing some trajectories of the game where orange is memory and dreaming. Up is good. So basically up represents game progress and down represents kind of game regression. And what's interesting is raw memory um basically without dreaming we can see crashes out a incorrect memory is planted and clock continues to basically regress or fall in the game when it hits that particular point with dreaming it rescues that mistake claw proceeds to the next level. That's really the key observation and this is a very general principle that claw can use dreaming to correct mistakes in memory that are otherwise stuck there. It's a very important way you can kind of um kind of correct from errors.

构建记忆表征：Wiki

这是在记忆中构建表征的另一个有趣例子。大家对将 wiki 作为 Claude 组织知识的通用方式表现出了浓厚兴趣。这是一个非常有趣的示例：我们在五月份收购了一家名为 Stainless 的公司，我们获得了该公司的语料库，包含其所有文档、Slack 消息等。我们对内容进行了去匿名化……我们对内容进行了匿名化处理，并基于它构建了一个数据集，用来测试梦境。我们所做的是创建了一个由 25 个关于该公司的问题组成的评估集，这些问题只能根据该语料库来回答。我们允许 Claude 根据这些信息构建一个 wiki，以帮助它回答问题。

英文原文

Now, this is another fun example of building representations in memory. So, there's been a lot of interest in wikis as kind of general um kind of ways that cloud can organize knowledge. This is a really fun example uh from we acquired a company Stainless back in May and we had access to the the company Corpus all its documents and Slack messages and so forth and we dean anonym we anonymize things and we built um kind of a a data set from it which allowed us to test dreaming. So what we did was we created kind of an email set of 25 questions about the company that can only be answered from this corpus. We allow Claude to build a wiki from this information to help it answer questions.

你可以看到，我们对比了一个原始搜索工具，它只是对这类文档进行某种语义搜索。我们将其作为基准进行比较，它的表现相当不错。你可以看到它达到了大约 53%。仅仅是搜索。我们还测试了让 Claude 在这套知识库上构建一个 wiki。你可以看到，随着你推进、投入越来越多的梦境 token，wiki 变得越来越好、越来越完善，使 Claude 能够回答问题。因此，Claude 原生具备随着时间推移，通过 wiki 逐步构建出对该知识语料库的表征的能力。随着你增加更多梦境计算，wiki 变得越来越有用。我们认为，对于思考智能体如何随时间推移构建知识，这里有一个非常好的心智模型：可以赋予它们访问文本转录记录原始语料库的权限，它们能够构建这些表征，以便更轻松地搜索这些转录文本来回答问题，并以有效的方式连接它们的知识。我们还发现，将 wiki 与搜索工具结合使用，实际上比单纯的原始搜索更进一步提升了效果。所以这带来了另一个直观认识：基本上 Claude 也可以在其记忆中构建像 wiki 这样的表征，以帮助它解决问题。

英文原文

What you see is we compare a raw search tool that's just like some kind of semantic search over this this kind of documentation. We compare that as a baseline which does reasonably well. You can see it's like 53%. Just for search. We also test letting quad build a wiki over this knowledge. And you can see as you push, you add more and more dreaming tokens, the wiki gets better and better and better allowing Claude to answer questions. So Claude natively can kind of build up over time a representation of this of this kind of corpus of knowledge through a wiki. And as you add more dreaming kapu, the wiki gets more and more useful. And we think there's a really nice mental model to think about how agents can kind of build knowledge over time. they can be given access to kind of raw corpuses of texts of transcripts and they can kind of build these representations that allow them to search those tonic texts much more easily to answer questions and to connect their knowledge in effective ways. We also find that adding the wiki and the search tool together actually bumps further beyond just raw search. So it's kind of another intuition that basically claude can also build representations like wikis in its kind of memory um in order to help it kind of solve problems.

最后一个例子来自一个不同的基准测试，但这里展示的是 Claude 构建的两个不同的 wiki。这是在梦境处理之前和梦境处理之后的对比。我们看到的是，在此特定案例中，当 Claude 构建 wiki 时存在一些空白，例如这个初始 wiki 包含了一堆针对不存在页面的占位符。当 Claude 构建各种记忆表征时可能会发生这种情况。如果缺少梦境计算，可能就没有机会去挽救或修复这些错误。我们观察到的是，当我们运行梦境时，梦境可以审视 Claude 现有的表征（例如 wiki），并能够填补疏漏。因此在这个案例中，它填补了许多缺失的空白。它创建了一个索引，因此再次证明，梦境是 Claude 进行反思、并在其记忆中投入计算以实现提升的一种方式——可以通过像我谈到的那样纠正错误，通过像我们在这里看到的填补其知识空白，或者通过创建像 wiki 这样的新表征来实现。

英文原文

And a final example, this is from a different benchmark uh but um this is showing two different wiks that claude built. So this is pri prior to dreaming and after dreaming. What we saw is when claude was building wiks in this particular case, there were some gaps and for example this initial wiki had a bunch of placeholders for pages that didn't exist. This can happen when cla's kind of building different memory representations. And without dreaming compute, there may not be opportunities to rescue or fix those errors. that we saw is when we ran dreaming, dreaming can look over Claude's existing representation like for example wiki and can fill in mistakes. So in this case it fills in a lot of missing gaps. It creates an index and so again dreaming is a way that claude can reflect and and kind of apply compute across its memory to improve it by correcting mistakes like I talked about by filling in gaps in its knowledge like we see here uh or for example by creating new representations like wikis.

嗯，讲到这里，我感谢大家的聆听，嗯，我接下来会在 Anthropic 的展台。所以，欢迎大家来找我，我很乐意交流关于这方面的任何话题。

[掌声]

英文原文

Um, so with that, I uh thank you all for listening and um I I'll be at the at the anthropic booth. So, please uh come see me and happy to talk about any of this.

[applause]

Syntax 播客开场

Heat. Heat. Heat. Heat. What Heat. Heat. Heat. Heat. Heat. Heat.

好了，各位。欢迎来到 Syntax。我和 CJ 在这里。我是 Scott Tolinski。这是 CJ Reynolds。我们将带来各种有趣的内容。我们将剖析一些最新的 AI 资讯，同时还会玩一些有趣的游戏，猜猜这是哪个 AI 模型做出来的。所以，如果你在现场，正在吃午饭或者做别的，你可以过来一起玩，或者就在那边放松、随便做点什么都行。不过，我们在线上也会和大家联动。CJ，近来如何，兄弟？在旧金山这边过得怎么样？

英文原文

Heat. Heat.

Heat. Heat.

What

Heat. Heat.

Heat. Heat.

Heat. Heat.

All right, folks. Welcome to Syntax. CJ and myself here. I'm Scott Tinsky. This is CJ Reynolds. We're going to be doing all kinds of fun stuff. We're going to be breaking down some of the latest AI news as well as getting into some fun games as to which AI model made this.

So, if you are out there, you're eating lunch or whatever, you can come hang out or you can just chill and just do whatever. But uh we'll be joining you online as well. So CJ, what's up man? How you doing in San Francisco here?

我很好。呃，我们来做个自我介绍，向那些还不认识我们的人介绍一下。我们是 Syntax。呃，我们是一档播客。我们还有一个 YouTube 频道，内容全都是关于 Web 开发、编程和 AI 的。呃，我们俩都来自丹佛。

英文原文

>> I'm good. Uh let's introduce ourselves and anybody that doesn't know who we are. We are Syntax. Uh we're a podcast. We also have a YouTube channel all about webdev programming AI. Uh we're both from Denver.

我们俩都来自丹佛。没错。

英文原文

>> We're both from Denver. Yes.

嗯，对，我想我们——

英文原文

>> Um yeah, I think we'll

是的。如今 Syntax 不仅仅是一档播客了。它还是一个直播。所以欢迎在 YouTube 上的 Syntax 关注我们。我们做各种各样的事情，包括很多像今天这样的内容。所以，

英文原文

>> Yeah. So Syntax is more than just a podcast these days. It's also a live stream. So check us out at uh Syntax on YouTube. We do all kinds of stuff including a lot of this. So,

最佳 Prompt 与隐式状态机

我们来梳理一下。呃，我们先来看第一个小环节，第一条新闻。他们称之为有史以来最棒的 prompt。这条推文最近在网上疯传，因为这确实是一个很好的 prompt：“找出我应用中所有隐式的状态机（implicit state machines）。”CJ，为什么这是一个好 prompt？你用过这个 prompt 吗？

英文原文

let's break it down here. Um, let's get into our first little bit, the first bit of news. They're calling it the greatest prompt ever. So, this tweet has been going viral here because this is a good prompt. Uh, find all implicit state machines in my app. CJ, why is this a good prompt? Have you used this prompt?

我没用过这个具体的 prompt，但我用过类似的。而且我认为这个 prompt 后面应该还有一个后续 prompt，因为隐式状态机本质上就是你没有显式定义的状态机，对吧？你的代码里散落着一堆变量，比如 isLoading 或者 isRunning，然后你在各处写满了针对这些变量的代码和 if 语句。所以这个 prompt 本质上就像是：让我们把那些面条代码（spaghetti code）转变成更容易推导理解的东西。对吧。

英文原文

I haven't used this prompt, but I've used similar prompts. And I think that like there would be a followup to this prompt because uh in and an implicit state machine is essentially a state machine that you haven't defined, right? So you have a bunch of variables all over your code like is loading or uh is running and then you have code and if statements all over those. And so this prompt is essentially like let's take that spaghetti code and turn it into something that uh is easier to reason about. Right.

是的。是的。所以基本上，如果你不知道什么是隐式状态机或显式状态机，最简单的理解方式就是：隐式状态机就是你以前在各种 React 代码库里经常看到被讨论的那些东西。像 isLoading、hasError、isSuccess 这些。大家以前常常把它们做成布尔状态，对吧？像这样。而这带来的大问题是，它容易出 bug、容易出错，而且还可能把你的 UI 置于不可能的状态（impossible states）。所以理论上，如果它们只是两个独立的布尔值，你可能会遇到 isLoading 和 isError 同时为 true 的情况，那这时候你是要把两者都显示出来，还是只显示其中一个或另一个？所以“找出我应用中所有隐式状态机”这个 prompt 的思路，就是你能把它们转换成显式状态机。显式状态机有声明好的状态。它们拥有的是具体的值而不是布尔标志位，对吧？并且它们具有状态转移，对吧，带有集中式的规则。所以如果你用过像 Redux 这样的东西——虽然它曾经一度……

英文原文

>> Yes. Yes. So basically, if you don't know what an implicit state machine or an explicit state machine is, the easiest way to think about this is that an implicit state machine is stuff that you used to see talked about all over React code bases. Stuff like is loading, has error, is success. Uh people these are often like a a boolean state, right? Like this. And the big problem with that is is that it can be bugprone, it could be errorprone, but it can also put your UI into impossible states. So the theoretically if it's just two separate booleans for these things, you could have a a is loading and an is error both true and then what you're showing both of them or you're showing one of them or the other one. So the the idea with this prompt find all implicit state machines in my app is that you are able to convert them to explicit state machines. Explicit state machines have a declared status. Uh they they have a value rather than a boolean flag, right? And they have state transitions, right, with centralized rules. So if you've ever used something like Redux, which at a time was

冗长，很多人因为这些原因不太喜欢它，但它最终成了实现显式状态机的一个极好构想。围绕显式状态机有很多库，比如 Redux……呃，不对，不是 Redux，呃，XState。

英文原文

>> verbose and many people didn't love it for those reasons, it ended up being a great idea for explicit state machines. Um there's a lot of libraries around explicit state machines like Redux um or not Redux uh X XATE.

XState，XState。是的。

英文原文

>> XATE XATE. Yes.

呃，这让 AI 辅助编程变得容易了一些，因为这些不可能的状态就不复存在了。然后你也可以在代码库周围做更多可视化。呃，你觉得在代码库周围做可视化有多大用处？

英文原文

>> Um and it makes working with AI coding a little bit easier because these impossible states are no longer possible. And then you can also visualize around your codebase a bit more. Um, how do you how how useful do you find visualizing around your codebase?

挺有用的，我是说，总的来说我确实觉得这很有用，而且这实际上是对很多人的有力反驳——那些人基本上在说：“别管代码了，让 AI 做所有决定。”是的。

英文原文

>> Yeah, I mean I like just in general though like I find this useful and it's actually an argument against a lot of people that are basically saying uh don't worry about the code, let the AI make all the decisions. Yes.

呃，但我发现，更整洁的代码库也让 AI 在其中工作变得更加容易。呃，所以如果你给它这样一个 prompt，就会更不容易出现像你刚才提到的那种怪异的 UI 状态。呃，我最近一直在用的一个 prompt 是：“找出所有的重复代码以及进行抽象的机会。”

英文原文

>> Um, but I have found that a cleaner codebase makes it e easier for the AI to work in as well. Uh, and so if you give it a prompt like this, it's going to make it harder to have like a weird UI state like you were mentioning. Um, one prompt that I've been doing recently is, um, find all duplication and opportunities for abstraction.

没错。

英文原文

>> Yes.

就像很长一段时间里人们常说的另一句话：“噢，复制粘贴比抽象更好。”但是当你面对一个庞大而蔓延的代码库，并且 AI 必须在所有这些代码中工作时，拥有单一真实来源（single source of truth）或者在代码库中拥有一致的做事方式，也会让 AI 轻松得多。

英文原文

>> And like that's another thing that for a long time people were saying like, oh, copy paste is better than abstraction. But when you have a massive sprawling codebase and AI is having to work in all of that, having a single source of truth or like one way to do things in your codebase also makes it a lot easier for the AI.

对。

英文原文

Yeah.

不，那里面最大的问题——倒不是说那是坏事，最大的问题在于如何让 AI 遵循……

英文原文

>> No, the the biggest problem with that, not that that's a problem, the biggest problem with that is getting the AI to adhere

遵循你的抽象规范，对吧？因为否则的话，你最终会得到好几份抽象的副本，或者……是的，

英文原文

>> to your abstractions, right? Because otherwise you'll end up with copies of abstractions or Yes,

没错。就我个人而言，我发现 lint 规则在处理这方面非常管用。比如，一个例子是我正在构建的一个 CLI 工具，它涉及到大量的文件读写。所以它会在你系统的各处写入文件并保存它们。在重构之前，它大概有 10 个不同的地方用了 10 种不同的方式来写入文件。但后来我把它们全部整合进了一个单独的文件管理器中，我的代码中任何想要写文件的部分都必须使用这个文件管理器。所以就有一条 lint 规则写着：禁止裸调 fs.read，禁止裸调 fs.write。这基本上就强制了 AI 必须使用那个抽象。

英文原文

>> exactly. And personally, I've found like lint rules to be really good for that. So like uh one example is a a CLI tool that I'm building that does a lot of file read and writes. So it's writing files all over your system. It's saving them. And before I refactored, it had like 10 different places with 10 different ways of writing the file. But then I brought it all into a single file manager and any part of my code that wants to write a file has to use the file manager. So there's a lint rule that says no bare fs read, no bare fs write and that basically forces the AI to use that abstraction.

完全赞同。是的。所以我很喜欢这个 prompt，也喜欢这类事情。因此，找出隐式状态机并将它们改成显式状态机，并在脑海中切实掌控好应用的状态，在我看来确实能够降低代码库的劣质程度（slop level）。

英文原文

>> Totally. Yeah. So uh I I I like this prompt and I like this type of thing. So finding your implicit state machines and changing them to be explicit state machines and like really getting a handle on the state of your application in my in my mind is something that does decrease the slop level of a code base.

确实如此，而且这里是 AI 工程师大会。所以我们依然需要软件工程。这不仅仅是 prompt 的事。

英文原文

>> Definitely and and this is the AI engineer uh conference. So like we we still need engineering. It's not just prompts.

是的。是的。是的。都是 prompt 啦，CJ。好吧。我们带来的下一件事是 unfair play。FairPlay 基本上就是 iOS 应用进行加密从而无法被反编译的方式。

英文原文

>> Yes. Yes. Yes. It is all prompt CJ. All right. The next thing we that we brought here is unfair play. So fair play uh is basically how a iOS app can be encrypted so that way it cannot be decompiled.

UnfairPlay 与应用修改

所以 UnfairPlay 会解密 IPA，它允许你恢复为 macOS 应用构建的实际应用程序。这个项目非常棒，因为它的做法有点像电子游戏领域的 ROM 破解圈子，在那里……

英文原文

>> So unfair play decrypts an IPA and it allows you to recover the actual application that was built uh for a Mac OS app. And so this project's really neat because what it's doing is it's it's kind of like the ROM hacking scene of of video games where

你并不是给别人提供假冒的应用，或者你没有在托管 YouTube 应用，但你潜在可以做的是，你可以拿到你自己下载的 YouTube 应用。你可以用 UnfairPlay 处理它。你可以拿到 IPA。你可以随心所欲地修改应用程序，然后重新安装到你的设备上。CJ，你觉得这听起来怎么样？

英文原文

>> you aren't supplying a a a counterfeit app to somebody or an app that is you're not hosting the YouTube app, but potentially what you could be doing is you could take your own YouTube app that you've downloaded yourself. You can unfair play it. You can get the IPA. You can modify the application to your heart's content and then reinstall it on your device. How does that sound to you CJ?

听起来太棒了。我是说，我是用 Android 的人。如果你听过 Syntax 播客，你就会知道这一点。是的。

英文原文

>> Sounds amazing. So, I mean, I'm an Android person. If you listen to the syntax podcast, you know that. Yeah.

而且我们一直都能做类似的事情。呃，在 Android 世界里有一个叫 ReVanced 的项目，非常类似。所以说，拿一个 Spotify 应用、拿一个 YouTube 应用、拿一个 Instagram 应用，然后移除其中向你展示广告的部分或诸如此类的东西，这是非常普遍的做法。倒不一定是在鼓励这么做，但确实有这么回事。

英文原文

>> And we've been able to do a similar thing. Uh, so there is this project called Revance in the world of of Android that is very similar. So, I mean, it's very common to take a Spotify app, take a YouTube app, take an Instagram app and remove the parts that show you ads or something like that. Not condoning that necessarily, but that's something.

不，把那些广告去掉。是的。不，我觉得我看到大家用 UnfairPlay 做的事情是，有人拿到 YouTube，然后把里面所有的 YouTube Shorts 统统移除了，说实话，这主意真不赖。是的。

英文原文

>> No, remove those ads. Yeah. No, and I I think what I saw with unfair play is that people were taking YouTube and then removing all YouTube shorts from it, which honestly that's not a bad idea. Yeah.

比如，想象一下修改 Instagram 让它给你设置个暂停时间。我……可能系统内置了一些诸如“不要刷太久”的规则，但它们并不总是起作用。比如你可以强制执行。你可以让它只在特定时长内显示信息流，然后你就必须把应用放下之类的。或者只是隐藏掉你不需要的东西，你知道的，应用里塞了太多功能，尤其是像现在的 YouTube，他们硬是把 Shorts 放进了主栏，还把订阅内容从主导航里移除了。所以，嘿，有了 iOS 上的 UnfairPlay，你就可以直接把它调换回去。

英文原文

>> I mean, that's like imagine modifying Instagram to give you a timeout. I There's probably some built-in rules that are like don't scroll for too long, but they don't always work. Like you could enforce it. You could make it so that it only shows you feeds for a certain amount of time and then you have to put the app down or something like that. or or just like hiding stuff you don't need, you know, like there's so many functionalities in apps, especially like YouTube now that they did put shorts in the main bar and they removed subscriptions from the main navigation. So, hey, with unfair play on iOS, you could just swap that back.

夺回属于你的应用。

英文原文

>> Take back your apps.

夺回属于你的应用。我们正在夺回我们的硬件。

英文原文

>> Take back your apps. We're taking back our hardware.

是的，

英文原文

>> Yeah,

我们正在夺回属于我们的应用。

英文原文

>> we're taking back our apps.

而且我……是的，而且这跟最近出现的像那个“逆向工程一切（reverse engineer everything）”的代码仓库超级契合。像这种……这才是 AI 带来的好的部分，就是像这类东西。所以是的，

英文原文

>> And I Yeah, and this is super relevant with like the the reverse engineer everything uh repo that came up. like this is this is the the good part of AI is yes stuff like this. So yeah,

掌控权和控制力，所有这些好东西。

英文原文

>> the ownership and the control, all that good stuff.

我全力支持这个。

英文原文

>> I'm here for it.

是的，我也是。好的，

英文原文

>> Yes, me too. Okay,

我们继续看一些新闻吧。你觉得呢？

英文原文

>> let's uh let's keep going on some news. How do you feel about that?

来吧。对，

英文原文

>> Let's do it. Yeah,

我们还为大家准备了一些编程游戏，但我觉得大家现在都在吃东西，所以可能不是介绍那些游戏的最好时机。好的，那么……

英文原文

>> we have some coding games lined up for you too, but I think people are eating right now, so maybe not the best time to introduce those. Okay, so

面向 Agent 的编程语言兴起

我们来聊聊面向智能体（agents）的编程语言的兴起。就在最近，来自 Anomaly（也就是 OpenCode 团队）的 Kit 一直在谈论他的新语言 Blood，而且除了几眼连他自己都形容为过时的语法之外，他其实还没真正展示过它。不过 Blood 以及许多其他这类 AI 语言，已经引发了这种讨论：AI 语言、AI 优先语言到底是不是值得探索、值得深入研究的方向？我们之前在 Syntax 上讨论过其中的几种，但现在除了 Blood，还有 bendylang.com 上的 Bend 语言，以及来自 Vercel 团队的 Zero 语言。所以我敢说，这只是我随口想到的三个。我大概已经见过 15 个这种家伙了。呃，在之前大家都是……

英文原文

>> let's do uh the rise of coding languages for agents. So very recently um Kit from the uh Anomaly which is the open code team has been talking about his new language blood and he hasn't shown it really beyond just a couple of glimpses of even what he had described as outdated syntax. Um but blood as well as many of these other AI languages have kind of turned into this conversation of our AI languages AI first languages really something that is worth exploring worth in worth investigating or whatever and so we've talked about a few of these on syntax before but now with blood there's also bend lang at bendylang.com as well as zero lang from the versel team. So there there's I I will say that those are just three that I I I got off the top of my head. I probably seen like 15 of these bad boys. Um so before it was everybody

是在搞一个 Safari 的 fork，或者一个浏览器 fork，或者 Chrome fork。

英文原文

is creating a Safari fork

or a browser fork or Chrome fork.

呃，然后大家都在搞前端语言之类的。现在我们有了 vibe-coded 语言，而且你最好相信，CJ，［笑声］它们全都比 Rust 快。你知道的，它们都很快，全都很神速。

英文原文

>> Um then everybody is creating a

front-end language or whatever. Now we

got vibecoded languages and you better

believe CJ [laughter] all of them are

faster than Rust. You know they're all

fast. They're all very fast.

那可都是它们的基准测试。

英文原文

>> That's their benchmarks.

没错。是的，我的意思是，我觉得这超级有意思。嗯，我的第一反应是，AI 是在现有的编程语言上训练出来的，那它为什么会在一种新编程语言上表现更好呢？但如果你看看他们的一些原则，以及他们究竟是如何让它对 AI 真正有用的，归根结底其实就是给它提供类似证明的东西，防止它犯错。呃，有点类似于 Elm 语言，就像有那么几种编程语言，让你很难陷入错误状态，或者出现内存泄漏之类的问题。

Elm 低调地在当下获得了大量的关注，原因涵盖方方面面。事实上 Effect 以及 Foldkit——也就是 Effect 里的 UI 框架——就是一种 Elm 风格的，我明白……

英文原文

>> Yeah. Yeah, I mean it's I think it's

super interesting. Um I mean my my

initial reaction to this is AI was

trained on existing programming

languages. So why would it be better at

a new programming language? But if you

look at some of their tenants and like

how they're actually making it useful

for AI, it really comes down to like uh

giving it like proofs and preventing it

from making mistakes. uh almost similar

to like Elm laying like there's various

programming languages that make it

really hard to get into error states or

have um like memory leaks and stuff like

that.

Elm Elm is lowkey uh getting a lot of

attention right now because of all kinds

of things and in fact effect and fold

kit which is the UI framework in effect

is an Elm uh style I see

框架，所以我看到 Elm 似乎成了如今在这个领域被讨论得相当频繁的事物之一。

英文原文

>> framework and so I I see Elm as one of

these things that seems to be talked

about quite a bit more in this space now

嗯，挺有意思的是你提到了 Effect，因为我并不把 Effect 当成一种编程语言，它更像是一种运行时，

英文原文

>> well it's interesting that you bring up

effect because I don't think of effect

as a programming language it's a more of

a runtime time,

但它确实为你提供了一些与这些语言相同的保障，对吧？比如，它让你拥有显式的错误状态，而且错误必须以某种特定的方式传递。嗯，再次强调，这只是让 AI 在那个代码库中编写代码时变得更加可预测。

英文原文

>> but it gives you some of these same

guarantees that these languages do,

right? Like it it um makes it so that

you have explicit error states and

errors have to uh pass through in a

certain way. Um and again, it just makes

AI more predictable when it's writing in

that codebase.

没错。是的。而且伴随着 Effect，还有一个基于 Effect 构建的前端框架叫做 Foldkit。Foldkit 背后的开发者刚刚宣布他将全职投入这项工作。现在，这玩意儿绝对厉害得很，原因有很多。你拥有单一状态树。它完全基于 Effect。一切皆为 schema。智能体非常擅长使用这个框架。

英文原文

>> Yeah. Yeah. And and along with effect,

there is a front-end framework called

Fold Kit that is built on effect. And

the uh the de developer behind Fold Kit

just announced that he's going to be

working on this full time. Now, this

thing absolutely rips for a lot of

reasons. You have one state tree. It's

all effect based. Everything's a schema.

Agents are good at this framework.

而且他们的前提并不是——你知道，对于很多前端框架、JavaScript 框架来说，总是“这个更快，这个比那个更快”或者别的什么。这个则是“这个是正确的，而且 AI 很擅长编写它”，并且我实际上看到了一些针对它的替代渲染器。是的。呃，有人正从一个 Foldkit 应用中获取 GPUI 输出。

英文原文

>> And their premise isn't, you know, with

a lot of front-end frameworks,

JavaScript frameworks, it was always

this is faster, this is faster than this

or whatever. This is this is correct and

the AI is good at writing it and your

and and I actually saw some alternative

renderers for this. Yeah. Uh, someone

was getting GPUi output from a Fold Kit

app.

哇。

英文原文

>> Wow.

所以直接奔着原生去了。

英文原文

>> So, going straight native on it.

我觉得那太重大了。这太有意思了，我的意思是，时代正在改变，因为换作以前我看到类似 Foldkit 的东西，会琢磨：好吧，我会用它吗？而且我是说，这也是我一直没上手 Effect 的原因之一，因为那玩意儿有点晦涩。比如我得花好一阵子才能搞明白：这东西到底在干嘛？是的。

英文原文

>> I think that's huge. It's so funny like

I mean times are changing cuz like I

would see something like Fold Kit and

think about it. Okay, would I use this?

And I mean that's one of the reasons I

haven't picked up effect because like

the the it's a little obtuse. Like it

takes a while for me to figure out like

what is this thing actually doing? Yeah.

但如果它是给智能体用的，

英文原文

>> But if it's for the agent,

如果它是给智能体用的，

英文原文

>> if it's for the agent,

那就没那么要紧了，只要它能让应用正确且更好就行。所以，

英文原文

>> then it doesn't matter as much as long

as it like makes the app correct and

better. So,

给智能体来点小甜头。

英文原文

>> give the agent a little treat.

是的。

英文原文

>> Yeah.

是啊。就是给智能体的一点小奖励。让它去搞搞什么小电脑竞争对手。谁知道呢，对吧？呃，我觉得 Foldkit 是个挺有意思的东西，因为我们过去讨论过的一件事——而且这还是过去，比如大概两个月前，所以已经是远古历史了——就是我不知道这些 AI 语言究竟能否流行起来，因为模型怎么可能写得出来呢？但在 Foldkit 和 Effect 这里你能得到的是，你把一个示例代码库扔进你的代码库里，你懂的，直接扔进去，然后你只要告诉智能体去参考这些示例、参考代码库，智能体处理起来完全没有问题。一点问题都没有。

英文原文

>> Yeah. Just a little treat for the agent.

Let him hack a little uh computer

competitor. Who knows, right? Um the

fulkit I think is an interesting thing

because one of the things that we we've

talked about in the past and this was

like in the past like what like two

months ago, so it's ancient history was

that I don't know if these AI languages

will ever catch on because what are the

models going to be able to write them?

But the thing that you get with fold

kitten in effect is that you get

You dump an example repo into your

codebase into just you know and you just

tell the agent to consult the examples

and consult the codebase and the agent's

got no problem with it. It's got no

problem.

所以依我的经验来看，我觉得那已经不再重要了。曾经有段时间，你确实得非常担心它有没有针对这个特定框架进行过训练。而且作为一个写了很长时间 SvelteKit 的人，那里面的一大痛点就是它不在训练数据里。所以，你因此不得不引入各种各样的确定性检查、技能以及 MCP 服务器来验证你的代码库。在某种程度上，现在也依然需要，

英文原文

>> So in my experience

I don't I don't think that matters

anymore. At one point in time you had to

really worry about did like was it

trained on this specific framework? And

as somebody who wrote speltkit for a

long time, the big problem with that was

is that it wasn't in the training data.

So therefore, you had to then

incorporate all kinds of deterministic

checks and skills and uh MCP servers to

validate your codebase. And you still do

to an extent,

对吧？

英文原文

>> right?

没错。但现在你可以直接把所有这些材料塞进上下文里，它就能写出来了。

英文原文

>> Yeah. But now you can just dump all this

stuff in context and it can write it.

而且上下文空间足够大。是的。我是说，我查了一件事，就是 foldkit.dev/llms.txt。它是存在的。所以你把它扔给智能体，它就能找到所有提供示例的页面，而且就像你说的，现在的上下文已经足够大了，

英文原文

>> And the context is big enough. Yeah. I

mean, one thing I checked is

fullkit.devlms.txt.

It exists. So you pop that into an agent

and it it finds all the pages that give

examples and uh like you were saying the

context is big enough now that

嗯，它可以把那些文档放进上下文里，并把事情做对，而不是去幻觉标签名，或者把 Svelte 5 写成 Svelte 3 之类的。

英文原文

>> um it can have those docs in context and

do the right thing instead of like

hallucinating tag names or or using

spelt 3 instead of spelt 5 or something

like that.

没错。对于 Foldkit 来说，他们甚至还有一个可以直接连接到他们开发工具的 MCP，这太酷了，因为智能体随后就能以一种面向智能体的方式来使用这些开发工具，对吧？

英文原文

>> Yeah. And so with with Fulkit, they even

have an MCP to connect directly to their

dev tools, which is sick because uh the

agent can then use the dev tools in an

agent focused way, right?

没错。

英文原文

>> Yeah.

我觉得这是个非常棒的亮点，它改变了我对智能体优先编程语言的看法，因为起初我会说：“是的，我不知道它们能否坚持下去。”嗯，但是看到在这上面开发东西是如此简单，而这显然不在任何训练数据或类似内容里。是的。呃，这确实彻底改变了我的看法。

英文原文

>> And I I think that's a really great

thing that that has changed my mind on

agentic first programming languages

because at first I would have said,

"Yeah, I don't I don't know if they'll

they'll hang in there." Um but seeing

how easy it was to develop something on

this which is clearly not any of the

training data or whatever. Yeah. Um it

has really changed my mind on this.

话虽如此，就像如今其他所有 vibe 编码的东西一样，存在着一场注意力争夺战。对吧。

英文原文

>> That said, just like with everything

else uh vibe coded these days, there is

a battle for attention. Right.

没错。关键全在于谁在用它、有多少人在用它，而不是……我的意思是，也许有朝一日我们所有人都会拥有各自喜欢的专属编程语言。是的。嗯，因为就像 Kit 针对 blood lang 所说的那样，语法完全可以是你想要的任何形式，虽然他还没发布太多……

英文原文

>> Right. It's all about who's using it,

how many people are using it rather than

I mean maybe maybe someday we'll all

just have our own programming languages

that we like. Yeah. Um because like Kit

said in regards to blood lang that the

syntax can be whatever you want it to be

which he hasn't released too much

意思是。好的。所以他只是在推文里聊了聊这个，因为我还没见过这东西。原来如此。［笑声］

英文原文

>> mean. Okay. So he's just talking about

it in his tweets because I haven't I

haven't seen this. So yes. [laughter]

所以关于 blood lang 这个东西，他确实还没正式宣布太多内容，但话说回来，我认为伴随着我们正在涉足的任何领域，智能体优先的开发工具都将带来极大帮助。但我同样认为非常有可能的是：对智能体而言最棒的语言，或许并不是 TypeScript、Rust 之类的，对吧？

英文原文

>> So the the bloodlength thing he hasn't

really announced too much of it but

again there are I I think with any of

the things that we're getting into

agentic first dev tools are going to be

really helpful. But I also think it's

very possible that the best agent

language for agents may not be

Typescript, Rust, or whatever,

对吧？

英文原文

>> right?

有可能。

英文原文

>> Possibly.

这一点，

英文原文

>> Which,

是啊，

英文原文

>> yeah,

对我来说并不是什么坏事。

英文原文

>> isn't that bad for me.

不，我的意思是，我也在为“拥有某种最喜欢的语言或框架”这种概念的消逝而感到惋惜。

英文原文

>> No, I mean, and I I I also I'm I'm

lamenting the death of having a favorite

language or framework.

是的。

英文原文

>> Yes.

语言就像是小吉祥物。我们了解它们。是的。

英文原文

>> Languages are like little mascots. We

get them. Yeah.

是啊。我是说，那是我们作为开发者的身份认同。呃，所以是的，现在轮到智能体了。应该由智能体来拥有一个偏好。比如如果……而且这甚至都谈不上是一种偏好。如果智能体在那种语言或框架下工作得很好，那大概就是你应该使用的工具。我是说，这也是我重新开始用 React 的原因之一，因为智能体非常擅长写 React。是的。

英文原文

>> Yeah. I mean, it was our identity as

developers. Um, so yeah, now it's it's

the agent. The agent should have a

favorite. Like if the And it's not even

a favorite. If the agent works well in

that language or framework, that's

probably the one that you should use. I

mean, it's one of the reasons I started

using React again is because the agents

are really good at React. Yes.

是的。而且你知道我想说什么吗？我确实用 Foldkit 开了一个新项目，

英文原文

>> Yes. And you know what I will say? I did

start a project with Fold Kit

而且我挺喜欢它的产出。所以，呃，我们拭目以待吧。是的，确实如此。

英文原文

>> and I kind of really liked its output.

So, uh, we will see. Yeah, for sure.

大家最先是在这儿听到的。你们最先在这儿听到的。Scott 不再用 Svelte 了。

英文原文

>> You heard it here first. You heard it

here first. No more spelt for Scott.

是的。对。［笑声］呃，不，那倒也不一定。只是觉得……

英文原文

>> Yes. Yeah. [laughter]

Well, no, that's not necessarily true. just felt

呃，但我当时碰到了问题，特别是在大型应用里的状态管理方面，而 Foldkit……

英文原文

>> uh but I was having problems

specifically around state in a large

application and foldit

基本上就像我们刚才聊到的，关于隐式状态机的问题。

英文原文

>> basically like we were talking about

with um uh implicit state machines.

没错。

英文原文

>> Yeah.

呃，Foldkit 让很多这类情况变得不太可能发生，

英文原文

>> Uh FKit makes a lot of that less

possible,

对吧？

英文原文

>> right?

嗯，因此，仅凭框架本身，你最终就能免费获得许多这类好处。好，我们来聊聊另一门编程语言，呃，

英文原文

>> Um so because of that uh you end up

getting a lot of those benefits for free

just based out of the framework. Okay,

let's talk about another programming

language um

Discord 转向 Rust 与 GPUI

那就是 Rust。

英文原文

>> which is uh Rust.

噢，

英文原文

>> Oh,

好。好的，我们提到了 Rust。嗯，Rust 写起来很棒，这太酷了。Discord 刚刚宣布他们正在将……

Discord 自身迁移到 Rust，并使用 GPUI 来实现。

英文原文

>> okay. Okay, we mentioned Rust. Um, Rust

being great to write is awesome. And

Discord just announced that they were

moving

Discord themselves to Rust and using

GPUi to do it.

难以置信。是的。有一条来自 Stanislav 的推文。嗯，他在 Discord 工作。他是……是的。

英文原文

>> Incredible. Yeah. So, there there's a

tweet from uh Stannislav. Um, he works

at Discord. He's Yeah.

嗯，他发了一张他 Windows 任务管理器的截图，代号为 Discord Phoenix 的版本只占用了 290 MB 内存。嗯，我的意思是，就像我们之前在播客里讨论过的那样，现在这个时代一切都在变得更快，并且正在用 Rust 重写。

英文原文

>> Um, and he he has a screenshot of his uh

like Windows task manager and Discord

Phoenix is the code name only using 290

megabytes of RAM. Um, and so I mean this

is we've talked about on the podcast

before like this is the age of

everything getting faster and being

rewritten with Rust.

嗯，所以我完全支持。我知道你用过相当多次 GPUI。嗯，但我觉得只要应用能跑起来，突然间我就多出了 1 GB 的空闲内存可用。我完全举双手赞成。

英文原文

>> Um, so I'm here for it. I mean I know

you've used GPUi quite a bit. Um, but I

think like as long as the app works all

of a sudden I've got a gig extra RAM to

use. I'm here for it.

没错，各位。那就让这成真吧。你们刚才听到了，Discord 正在转向……

英文原文

>> Yes, folks. So let this be a thing. You

just heard that Discord is moving to

GPUi。我已经谈论 GPUi 好几个月了，好多个月了。你之前也听我提到过 Fold Kit。大概六个月后，当其他人开始注意到的时候，我认为 Foldit 到时会在这里获得一些重大报道。

英文原文

GPUi.

I've

been

talking

about

GPUi

for

months,

for

many

months

now.

And

you

heard

me

talking

about

Fold

Kit

earlier.

In

about

about

six

months,

when

other

people

start

to

catch

on,

I

think

Foldit's

going

to

be

getting

some

major

press

here.

只要听播客，听斯科特的，你就会知道接下来会发生什么。基本上，

英文原文

>> Just

listen

to

Just

listen

to

the

pod,

listen

to

Scott,

you'll

know

what's

coming

next.

Basically,

syntax.fm，各位，我们每周一都有直播。呃，来看看我们吧。所以，你觉得这个显然是一个智能体驱动的决定，对吧？嗯，Discord 是那种每次打开……或者说是每次打开都会重新安装的应用，对吧？总是有更新要来。所以，嗯，这也是一个我从来没觉得性能低下的应用。我订阅了——

英文原文

>> syntax.fm,

we're

live

every

single

Monday,

folks.

Uh,

check

us

out.

So,

how

how

do

you

feel

about

this

is

clearly

an

agentic

decision,

right?

Um,

Discord

is

one

of

those

apps

that

opens

every

sing

or

like

re

uh

reinstalls

itself

every

single

time

you

open

it,

right?

There's

always

an

update

coming.

So,

um,

it's

also

an

app

that

I've

never

thought

was

non-performant.

I

I

subscribed

to

大概有 40、50 个 Discord 频道吧。

英文原文

>> maybe

like

40

50

Discord

channels.

是的。

英文原文

>> Yeah.

我一直对它处理得如此出色感到惊讶。话虽如此，他们能够用 GPUi 重写整个应用程序——Discord 可不是个小应用——这意味着，天哪，我们现在可以轻松地针对性能进行优化了。

英文原文

>> And

I've

always

been

surprised

at

how

well

it

handles

that.

That

said,

the

fact

that

they

were

able

to

rewrite

the

entire

application,

Discord,

no

small

application

in

GPUi,

means

that

like

man,

we

can

optimize

now

easily

for

performance.

确实。而且我的意思是，你想想看，为什么花了这么长时间才做这件事？比如他们使用 Electron 的原因之一就是因为它是跨平台的。对。他们写一个应用，就能在 Windows、Mac 和 Linux 上运行。他们拥有庞大的用户群体，

英文原文

>> Definitely.

And

I

mean,

you

you

think

about

like

why

did

it

take

so

long

to

do

this?

Like

one

of

the

reasons

they

were

using

Electron

is

because

it's

crossplatform.

Yes.

Right.

They

write

one

app.

It

works

on

Windows,

Mac,

and

Linux.

They

have

a

massive

user

base,

分布在各个地方。

英文原文

>> distributed

anywhere.

是的。所以 Electron 在这方面很棒。我的意思是你也谈到过 Tauri 的一些痛点。

英文原文

>> Yeah.

And

so

Electron

is

great

for

that.

And

I

mean

you've

talked

about

some

of

the

woes

of

Tari.

是的。

英文原文

>> Yes.

比如它类似于 Electron，但速度会稍微快一点，并且它自带系统 web view，但是你会遇到很多潜在的兼容性问题，比如 macOS 上的 WebKit 对比 Linux 上的 Chrome。嗯，而在 Electron 中，你就不会遇到这种情况。所以，我理解为什么这么长时间以来一直是 Electron。

英文原文

>> Like

it's

similar

to

Electron,

but

it's

going

to

be

a

little

bit

faster

and

it

ships

with

the

system

web

view,

but

you

run

into

a

lot

of

potential

compatibility

issues

with

like

WebKit

on

Mac

OS

versus

like

Chrome

on

Linux.

Um,

and

with

Electron,

you

don't

get

that.

So,

I

understand

why

it

was

Electron

for

so

long.

噢，是的。而且如果你把 Electron 和其他任何方案做对比，出于多项原因，Electron 仍然胜出。我的意思是，你可以在 Electron 中通过 flags 精确选择你的 Chrome 版本。所以是的，然后每个用户都会得到那个版本，但这也是你在 GPUi 中所能得到的，对吧，在 GPUi 中你拥有一个代码库，然后你会得到一个跨平台的原生应用，它在每个平台上都使用原生 UI 元素。

英文原文

>> Oh,

yeah.

And

if

you

were

talking

Electron

versus

anything

else,

Electron

still

wins

for

a

number.

I

mean

you

can

choose

your

exact

uh

Chrome

version

with

flags

uh

in

Electron.

So

yes

and

then

every

user

gets

that

but

that's

what

you

also

get

with

GPUi

right

you

have

one

one

codebase

with

GPUi

and

then

you

get

a

crossplatform

app

that

is

native

it

uses

native

native

UI

elements

on

on

every

platform

嗯，所以现在是做这件事的时候了，就像我们看到 Bun 用 Rust 重写了一样。是的。呃，几个月前我们在播客上讨论过这个，但呃，Jared 有一整套智能体流程，比如提取现有的代码库和测试，然后逐渐移植它们，逐渐让它们通过测试。嗯，我不确定他们是否能对 Electron 应用及其测试做类似的事情，因为它是非常偏向浏览器的，但与此同时，智能体在理解大型代码库并将其转换成其他语言或移植到另一种语言方面已经变得非常擅长了。是的，我确实想知道在测试方面的可移植性是如何运作的，他们的测试基础设施，比如他们是否转换了测试？是的，我、我真的很想知道这一点。那真的很有意思。除此之外，嗯，Anomaly 还重新发布了 Rune。你看过 Rune 吗？

英文原文

>> um

so

now's

the

time

to

do it

like

we

saw

bun

was

rewritten

in

Rust.

Yes.

uh

and

uh

there

we

talked

about

it

on

the

pod

a

few

months

back

but

uh

Jared

had

an

entire

agentic

flow

of

like

take

the

existing

codebased

and

tests

and

then

gradually

port

them

gradually

get

them

passing.

Um

I'm

not

sure

if

they

could

do

a

similar

thing

with

an

Electron

app

in

its

tests

because

it's

very

like

browser

focused

but

at

the

same

time

agents

have

gotten

really

good

at

like

taking

a

large

codebase

and

making

sense

of

it

in

some

other

language

or

or

porting

it

to

another

language.

Yeah,

I

do

wonder

what

that

how

how

that

portability

worked

with

testing

and

how

their

testing

infrastructure

like

did

they

convert

the

test?

Yeah,

I

I

I

really

am

curious

about

that.

That's

really

interesting.

And

on

top

of

this,

um

Anomaly

also

re

released

Rune.

Have

you

seen

Rune?

Rune。这是什么？

英文原文

>> Rune.

What

is

this?

R-U-N-E，就像你通常拼写 Rune 那样。呃，Rune 是他们的 GPUi 竞争对手。

英文原文

>> R

U

N

E

as

you

typically

spell

Rune.

Uh

Rune

is

their

GPUi

competitor.

它是一个基于 Rust 的 GPU 框架。这里的理念同样是它看起来会很棒，除了看到它发布了，并且知道它是一个用于 GPU UI 工作的不同 Rust 框架之外，我还没有深入去研究它，但是——

英文原文

>> It

is

a

Rustbased

GPU

framework.

And

the

idea

here

is

again

that

it's

going

to

look

nice

and

I

haven't

dove

into

it

beyond

just

seeing

that

it

was

released

and

that

it

was

a

different

rustbased

framework

for

GPU

uh

UI

work

but

好吧，所以他们要用这个重写像呃 open code 桌面版，或者我猜他们已经用这个构建了 open code 桌面版，现在正把它作为开源发布给其他人使用。

英文原文

>> okay

so

they're

going

to

rebuild

like

uh

open

code

desktop

or

I

guess

they

have

built

open

code

desktop

with

this

and

now

they're

releasing

it

as

open

source

for

other

people

to

use.

是的。嗯，我们上周谈到的那个应用。

天哪。Hex。

英文原文

>> Yes.

Um

that

app

that

we

talked

about

last

week.

Oh

man.

Hex.

Hex。是的。

英文原文

>> Hex.

Yeah.

我有 Dex，我有 Rex，我有 Hex，我的设备上有所有这些不同的应用。但是呃，那是用 Rune 构建的，好的，他们当时鼓吹了一些能力，可以让它看起来很棒，并且能很好地融入系统，而且呃在所有平台上都拥有良好的 UI，

英文原文

>> I

got

Dex.

I

got

Rex,

I

got

Hex,

I

got

all

these

different

apps

on

my

uh

thing.

But

uh

that

was

built

in

Rune

and

okay,

they

were

touting

some

of

the

abilities

to

have

it

look

nice

and

fit

in

and

uh

have

good

UI

across

all

platforms,

对吧？

英文原文

>> right?

并且让做到这些变得简单。所以，如果你想知道如何用 Rust 让跨平台应用保持高性能，这也是另一个值得关注的方案。嗯，但显然有些方案会涉及 web 领域，然后你就是在 Canvas 上渲染，那时候你根本没有使用 DOM 元素。所以它、它并不是——

英文原文

>> And

and

make

it

easy

to

do

so.

So,

this

is

another

one

to

keep

your

eye

on

if

you

are

wondering

about

making

crossplatform

apps

performant

with

Rust.

Um,

but

obviously

you

get

into

the

web

space

with

some

of

these

and

then

you're

you're

what

you're

rendering

on

a

canvas

and

you're

not

using

DOM

elements

at

that

point.

So

it

it's

not

你知道，存在所有这些不同的“一次编写，到处部署”的解决方案，无论是像 React Native 进行转换然后 React Native web 再转换回 web，还是诸如此类的任何东西，或者是把 web 内容拿来在手机中作为 web view 渲染等等。

英文原文

>> you

know

there

's

all

these

different

write

once

deploy

anywhere

solutions

whether

it's

like

React

Native

translating

it

and

then

React

Native

web

translating

it

back

to

web

or

any

of

that

or

taking

web

and

rendering

it

as

a

web

view

in

a

phone

or

whatever.

还有 Flutter，这也是另一个。

英文原文

>> And

there's

Flutter

is

another

one.

还有 Flutter。Flutter 那些人非常呃，如果我们再次忘记 Flutter，他们会让我们知道的，因为我觉得我们忘记 Flutter 的次数太多了。［嗤笑］但是，一直以来不一定真正存在的，是真正做到“一次编写，到处部署”，并且以毫不费力的方式在所有地方真正实现原生化。我认为除了 Flutter 之外，我们最接近这一目标的就是像 React Native、React Native web 的世界。但是 React Native 是一个很难、很难啃的平台，我们看到人们一直在离开它，对吧？

英文原文

>> There's

Flutter.

The

Flutter

folks

are

very

uh

they

will

let

us

know

if

we

forget

Flutter

again

because

I

think

there's

too

many

times

we've

forgotten

Flutter.

[snorts]

But

the

what

there

hasn't

necessarily

been

is

truly

the

right

once

deploy

anywhere

and

have

that

actually

be

native

everywhere

in

a

way

that's

effortless.

I

think

the

closest

that

we've

gotten

with

this

besides

Flutter

is

like

the

React

Native

React

Native

web

world.

But

React

Native

is

a

a

tough

a

tough

platform

and

we're

seeing

people

move

off

of

it

all

the

time,

right?

是的。我的意思是，再一次，我从来没想过自己会看到这一天，但这就类似于……你像以前一样，Electron 如此受欢迎的原因是 web 开发者已经熟悉它，它让跨平台变得超级、也让跨平台变得超级简单，但现在如果是由智能体在编写代码，而且性能很好，那为什么不呢？为什么不用原生？

英文原文

>> Yeah.

I

mean,

again,

I

I

never

thought

I'd

see

the

day,

but

it's

like

you

like

again,

the

reason

Electron

was

so

popular

is

like

web

devs

already

knew

it

made

it

super

and

also

made

it

super

simple

to

be

crossplatform,

but

now

if

the

agents

writing

the

code

and

it's

performant,

why

not?

Why

not

native?

为什么不？

英文原文

>> Why

not?

是啊，

英文原文

>> Yeah,

为什么不呢？是的，我知道。我完全同意这一点。嗯，我、我觉得其实还有另一个方案，我刚刚看到有人在讨论它是一个“一次编写到处部署”的方案。我正试着把它找出来，但我找不到。它刚刚在大约 12 小时前才宣布。

英文原文

>> why

not?

Yeah,

I

know.

I

totally

agree

with

that.

Um,

I

I

think

there

was

actually

there

was

another

one

that

I

just

saw

somebody

talk

about

that

was

a

right

ones

to

play

everywhere.

I'm

trying

to

pull

it

up,

but

I

cannot.

It

It

just

was

announced

like

12

hours

ago.

它是基于 Rust 的吗？还是……

英文原文

>> Was

it

Is

it

rust

based

or

噢，CJ，问得好。我不知道。［笑声］我不知道。

英文原文

>> Oh,

CJ,

that's

a

good

question.

I

don't

know.

[laughter]

I

don't

know.

东西太多了。一直以来发布的该死的东西实在太多了。

英文原文

>> There's

too

many

things.

There's

too

many

dang

things

released

all

the

time.

先放那儿吧。也许你待会儿能想起来。

英文原文

>> Let

it

sit

there.

Maybe

it'll

come

to

you.

观众提问：大厂内耗与工程师技能退化

好的。是的，完全没错。好的，很酷。嗯，让我们进入一些提问环节。我们在 Syntax 这边收到了大量的问题。所以我想深入探讨一些从整个 Web 开发和通用开发生态系统中涌现出来的问题。我们收到了许多这样的问题。我们收到的第一个问题是：“你们大家对大厂里的内耗/技能退化怎么看？我们是否认为大公司的软件工程师在不知不觉中丧失技能？”我今天在演讲中也大致谈到了这一点。我今天做了一个演讲。我们现在在 AI Engineer 大会。显而易见，人们正在失去技能。事实上，在我做的一项调查中，我认为大约有 67% 的人表示，他们觉得自从使用 AI 以来，自己的技能在减退。但我认为更大的问题不是“我们、我们在编码方面的技能是否在减退？”更大的问题是“那有关系吗？”

英文原文

>> Yes.

Yeah,

absolutely.

Okay,

cool.

Um,

let's

get

into

some

questions.

We

we

get

a

ton

of

questions

over

here

at

syntax.

So

I

want

to

dive

into

some

questions

that

are

coming

in

across

the

web

development

and

the

development

ecosystem

overall.

So

we

had

a

number

of

these

questions

come

in.

First

one

that

we

had

come

in

was,

"How

are

y'all

feeling

about

brain

rot

in

big

tech?

Do

we

think

software

engineers

at

big

companies

are

losing

their

skills

without

knowing

it?"

And

I

kind

of

talked

about

this

in

my

talk

today.

I

did

a

talk

today.

We're

at

AI

engineer.

I

obviously

people

are

losing

their

skills.

In

fact,

in

my

survey

that

I

did,

I

think

it

was

like

67%

of

people

listed

that

they

feel

like

their

skills

are

diminishing

since

using

AI.

But

I

think

the

bigger

question

isn't

do

are

we

are

our

skills

in

coding

diminishing?

The

bigger

question

is

does

that

matter?

是的。

英文原文

>> Yes.

对。我的意思是，再一次，我得收敛一下我过去说过的话，比如我们的技能组合正在转变，它们确实如此。就像你谈到惋惜，或者你甚至可能没想那么多，但如果你在这个领域待得足够久，事情就会发生变化，变化速度很快，特别是如果你处于 Web 开发领域，比如新的 API、新的浏览器，浏览器被废弃。所以，如果你曾经不得不呃为 Internet Explorer 6 实现过某些功能，你现在完全没有理由再去懂得怎么做了。它已经不再是一回事了。所以它正逐渐变成——

英文原文

>> Right.

I

mean

to

again

I'm

like

biting

my

tongue

on

stuff

I've

said

in

the

past

like

our

skill

sets

are

shifting

they

they

really

are

and

like

you

you

talked

about

like

lamenting

or

you

maybe

didn't

even

think

about

it

that

much

but

if

you've

been

in

this

space

long

enough

like

things

change

and

things

move

fast

especially

if

you're

in

the

web

dev

space

like

new

APIs

new

browsers

browsers

get

deprecated

and

so

if

you

ever

had

to

uh

implement

something

for

Internet

Explorer

6

there

is

no

reason

for

you

to

know

how

to

do

that

anymore.

it

is

like

no

longer

a

thing.

And

so

it's

gradually

becoming

那么现在我们的专业能力应该是什么？比如什么、什么——

英文原文

>> well

what

is

what

should

our

expertise

be

now?

Like

what

what

如果不再是编程，那么我们现在是编排者吗？我们现在是像技术负责人那样吗？嗯，所以我确实看到我们越来越朝那个方向发展。嗯，与此同时，要成为一个真正出色的架构师或技术负责人，你多少需要了解底层是如何运作的，这样才能做出良好的决策并给出正确的指导。

英文原文

>> what

like

if

it's

not

programming

then

are

are

we

now

u

orchestrators?

Are

we

now

like

tech

leads?

Um

and

so

I

I

do

see

us

moving

more

and

more

towards

that.

Um,

at

at

the

same

time,

like

in

order

to

be

a

really

good

architect

or

tech

lead,

you

kind

of

need

to

know

how

things

work

under

the

hood

to

be

able

to

come

up

with

good

decisions

and

have

good

guidance.

智能体仍然会随心所欲地做任何它想做的事，

英文原文

>> The

agents

will

still

do

whatever

the

hell

they

want,

对吧？我的意思是，而且我在我现在正在开发的工具上也发现了这一点，就像它想做什么就做什么，但然后我、我把它拉回来。我会说，“嘿，我们需要在这边稍微提高一点性能，或者我们需要稍微多共享一点代码。”我知道这一点是因为我自己曾经在大型代码库中工作过。所以，像这类的技能绝对能够迁移。

英文原文

>> right?

I

mean,

and

I

found

that

with

the

the

tool

that

I'm

working

on

right

now,

like

it

does

whatever

it

wants,

but

then

I

I

rein

it

back

in.

I'm

like,

"Hey,

we

need

to

be

a

little

bit

more

performant

over

here,

or

we

need

to

share

code

a

little

bit

more."

And

I

know

that

because

I

myself

have

worked

in

large

code

bases.

So

like

those

kinds

of

skills

definitely

translate.

技能演变与 AI 时代的“脑腐”

嗯，至于这种“脑腐”（思维退化）到底发生在哪里，我认为它存在于多个层面。很多其实源于管理层，比如管理层会要求“使用 AI，加快进度”，然后开发者就不得不紧跟节奏，接着他们就会精疲力竭，以至于现在他们甚至都顾不上担心自己是否还懂得写代码了。

英文原文

Um in terms of where the brain rot is happening like I think it's at multiple levels though because a lot of it comes from management like management is like use AI move fast and then developers are having to stay caught up and then they're burning out and so now they can't even worry about uh do they still know how to code.

嗯，所以这种情况确实在发生，但我认为这全都是这场大转变的一部分，就像是我们在朝哪里走、我们正走向何方？

英文原文

>> Um so it's happening but it's all I think part of this big shift of like where where do we where are we going where are we headed?

是的。

英文原文

>> Yes.

是啊。对。好的。所以，我想说一件事，而且我在演讲中也提到了这一点。那么，我想说有些事情我过去在 Web 开发中经常做。我过去了解针对 Internet Explorer 5、Internet Explorer 6 的每一种浏览器 hack，而我已经不再需要……我过去能够手写 Angular 组件。

英文原文

>> Yeah. Yeah. Okay. So, a thing I will say and I said this during my talk. So, uh I will say there are things that I used to do in web development. I used to I used to know every browser hack for Internet Explorer 5, Internet Explorer 6 and I haven't had to you I used to be able to write Angular components by hand.

是的，我也是。我已经太久没有做过这些了，以至于我的大脑在这些技能上都已经退化了。那么，这现在还重要吗？已经不重要了。我再也不用为 Internet Explorer 5 制作网站了，对吧？所以那些技能已经消失了。你可以为失去这些技能而叹息、感伤，但归根结底，时代已经变了，

英文原文

>> Yeah, me too. I haven't had to do this in so long that my brain in it has atrophied in those skills. So like does that matter anymore? It doesn't matter. I don't have to produce a website for Internet Explorer 5 ever again, right? So those skills are gone. And uh some of those things you can lament and you can mourn the loss of those skills, but at the end of the day, things have shifted,

对吧？所以是的，情况已经变了，而且我认为关键越来越在于什么才应该是重要的。比如，我绝对不认为我们应该仅仅沦为 prompt 撰写者，整天只会 prompt、prompt、prompt。那个 prompt 背后必须得有深思熟虑。是的。而这正是技能正在转移的方向。所以我确实认同，在那个灰色地带确实存在所谓的“脑腐”。比如人们会觉得“我现在所做的一切就是写 prompt，我根本不需要去操心那些”。我认为如果你想要做出好产品、想要带好团队，你依然应该关注那些东西。所以……

英文原文

>> right? And so yeah, they've shifted and I think it more it becomes what should be important. Like I I absolutely don't think it should be we are just prompters just prompt prompt prompt. There's has to be some thought behind that prompt. Yes. And that's where the skills are shifting to. And so I do agree that there is actually brain rot in that gray area. Like people like all I do is prompt now. I don't actually have to worry about. I think you still need if you want to have a good product, if you want to have a good team, you should still care about those things. So

嗯，对啊。是啊。因为归根结底，这其实是在寻找一种简单的解法，因为我们所看到的，尤其是随着 vibe coding 的兴起，就是人们直接把各种天知道是什么的东西一股脑塞进浏览器里，然后说“哎呀谁在乎呢，应用运行得挺好，Lighthouse 得分也还过得去”。结果你仔细一看，整个 DOM 被复制了两份，桌面端一份，移动端一份，你直接就傻眼了：“我们这到底是在搞什么啊？”你知道，那些底层的东西显然依然很重要。

英文原文

>> Well, yeah. Yeah. Because at the end of the day it is looking for a simple solution at the end of the day because what we saw especially with the um the rise of vibe coding was that people were just hucking all kinds of stuff god knows what into their their browser and you say oh who cares the app works fine it gets a decent lighthouse scores and then you look at it and the entire DOM is duplicated twice once for desktop and once for mobile and you're just like what are we doing here you know that that stuff still matters for sure

它依然很重要。

英文原文

>> it still matters.

是的。没错。

英文原文

>> Yeah. Yeah.

Vibe Coding 的定义与演变

太棒了。好的，我们来看下一个问题。能不能谈谈在 2026 年 vibe coding 究竟意味着什么？作为一名中级开发者，我看到大多数同行（如果不是全部的话）都在使用 AI，因为不用它就根本无法竞争。但对许多人来说，vibecoded（vibe 编码生成的）现在成了一个贬义词。哇。是的。有时它会被不公平地扣在纯手写项目、有经验开发者借助 AI 完成的项目上，而用在非程序员使用昂贵模型做出的东西上则是名副其实的。那么……

英文原文

>> Cool. All right, let's next question here. Uh, just could you say a few words about what vibe coding actually means in 2026? As a middlele developer, I see most if not all others using AI as it's impossible to compete without it. But for many vibecoded is now a slur. Wow. Yeah. uh sometimes unfairly thrown at handwritten projects, projects with AI by experienced devs and fairly non-coders and with expensive models. So

vibe coding 到底是不是个坏词？是个带有感情色彩的词吗？它在 2026 年意味着什么？它过时了吗？这到底是怎么一回事？

英文原文

>> vibe coding, is it a bad term? Is it a loaded term? What is what does it mean in 2026? Is it a dead term? What what's up with that?

我的意思是，我对这个词最初的本能反应是，它代表一种非常特定的 AI 编程方式：你根本不在乎代码，全凭感觉（vibes）。但我最近在做的编程，比如我也会称之为 vibe coding，因为所有代码都是 AI 写的。是的。但正如我之前谈到的，所有的决定依然是由我来做的。我依然在进行编排，我依然在进行工程构建。AI 工程。

英文原文

>> I mean, my my initial gut reaction to the term was like it's it's a very specific kind of AI coding where you don't care about the code and it's all about the vibes. But the coding that I've been doing lately, like I I would call it vibe coding because I'm AI is writing all of the code. Yes. But again, like I talked about earlier, I'm still making all of the decisions. I'm still orchestrating it. I'm still uh engineering it. AI engineering.

是的。没错。确实。对。

英文原文

>> Yeah. Yeah. Yeah. Yeah.

我依然在……你依然必须这样做。

英文原文

>> I am still like you still have to.

所以是的，我不会把用我正在构建的工具所做的事情称为 vibe coding。我会称之为 AI 辅助工程。不过，我确实看到 vibe coding 被当作贬义词四处乱用。就像有人发布了自己的应用，你就会说：“噢，那是 vibe code 出来的，不用在意。”

英文原文

>> So yeah, I I don't I wouldn't call what I'm doing with the tool that I'm building vibe coding. I would call it AI assisted engineering. Um, but I do see like vibe coding being thrown around as a slur. Like you can just you somebody posts their app, you're like, "Oh, that was vibe coded. Don't care about it."

所以这很棘手，因为我想说这其中的界限很难把握。

英文原文

>> So it's tricky because it's there's a it's a tough line to walk, I would say.

是的。确实。这很有意思，因为看起来确实有这么一群人，只要任何东西在任何环节有 AI 参与，就会被贴上“vibe 编码生成的垃圾”（vibecoded slop）之类的标签。直接扔进垃圾桶。我认为任何在现代语境下使用过这些模型的人——这里的“现代”就像过去的两周一样，谁知道这个行业现在变成什么样了对吧——任何以这种方式使用模型的人都知道，时代已经变了，模型也变得更好了。所以我不知道，对我来说，人们对这些术语如此纠结让我有点烦躁，但我明白它们为什么存在。我不能理解的是，为什么非要仅仅因为使用了 AI 就全盘否定一切。话虽如此，当出现明显的 AI 劣质痕迹时，确实有些东西是很糟糕的。是的。

英文原文

>> Yeah. Yeah. It is funny because it does seem like there's a group of people with if anything had AI had a hand in it at any point, it's labeled as vibecoded slop or whatever. Toss it in the bin. um which I think anybody who's been using any of the models in a modern context like modern isn't like the past you know two weeks yeah who who knows in this industry anymore right uh anybody who's been using models in that regard knows that like hey things have shifted and the models have gotten better so I don't know yeah to me like there has like the language doesn't like kind of bugs me a little bit that people get so hung up on the terms or whatever but I understand why they exist. What I don't get is like the insistence of throwing everything out just because AI is used. That said, there are some things when you have clear tells of AI that are bad. Yeah.

比如那些展现出缺乏用心、缺乏品味、对自己在做的事情缺乏掌控力的地方。我认为这类问题表现得非常明显，会瞬间让人大倒胃口。我都数不清如今我打开过多少个 YouTube 视频，看了大概两分钟就心想：“噢，这是一个 Claude 写的脚本。关掉吧，我不看了。”

英文原文

>> Like that that show a lack of care, show a lack of taste, show a lack of control over what you're doing. And those those types of things I I think stand out in a way that really turns people off instantly. I would not tell you how many YouTube videos I turn on these days and I get like two minutes into it and I'm like, "Oh, this is a this is a Claude script. Toss this. I'm done with this."

他们在照着 Claude 念。他们的动画也是由 Claude 生成的。但是之后没有任何跟进，对吧？没有剪辑，没有调整，没有把它变成自己的东西。是的。

英文原文

>> They're reading from Claude. Their animations were generated from Claude. But then there there was no follow-up, right? There's no editing. There was no tweaking, making it their own. Yeah.

而且 Claude 写的脚本也确实有点让我烦躁。并不是说 Claude 有问题，而是 AI 生成的脚本让我反感。因为当你能看出某人在念一段自己完全没有参与创作的脚本时，感觉就像是：既然你自己都不在乎，我为什么还要费心思去在乎呢，对吧？是的，

英文原文

>> And and and the script from Claude really kind of bugs me, too. Not that Claude's the problem. The script from AI bugs me. Um because when you can see somebody reading a script that they didn't have any hand in creating, it's just you you it it feels like why am I bothering to care about this if you didn't care about this, you know? Yeah,

确实是这样。

英文原文

>> definitely.

现场互动游戏：猜猜是哪个 AI 模型生成的

好的。那么，我们接下来要玩个简短的小游戏。我们来玩个游戏吧，CJ。

英文原文

>> Yes. Okay. So, we're going to play a quick little game here. Let's do a game, CJ.

好的，我准备好了。

英文原文

>> Okay, I'm ready.

好的，

英文原文

>> Okay,

各位。我们要玩个游戏了。如果有路过的人想参加这个游戏，请随时加入。如果没有，那就只有 CJ 一个人玩，那也完全没问题。我们要玩的游戏是“猜猜这是哪个 AI 模型制作的”。你可以在任何时候加入，也可以中途加入游戏。看看你能不能猜对。我们上面准备了几个。那么，CJ，你准备好了吗？

英文原文

>> folks. We're going to be doing a game. So, if anybody walks by, they want to play in this game, please join in. If not, it's just going to be CJ. That's all good. We're going to be playing Guess Which AI model made this. You can join at any given time. You can join in the middle of the game. Let's see if you can do it. Um, we got a few of these up here. So, CJ, are you ready?

开始吧。

英文原文

>> Let's do it.

好的，台下的观众们，如果你们想玩，可以扫描这个二维码。大家随时都可以加入进来。我们先从这里的第一个开始。哎呀天哪，我的 Wi-Fi 断了。好的，让我们打开这个家伙。

英文原文

>> Okay, audience, if you are out there, you all want to play. Scan this QR code. You can join at any given time. We're going to open this up with the very first one here. Oh, brother. I lost my Wi-Fi. Okay, let's open this bad boy.

我看到了。好的，它正在缓慢加载。哪个按钮是 Opus 5.5 做的？

英文原文

>> I see it. Okay, so it's uh it's slowly loading. Uh, which button is Opus 5.5?

好的。这些按钮中哪一个是 Opus 5.5 做的？

英文原文

>> All right. Which of these buttons did Opus 5.5 make?

好的。选项 A，我看到类似绿色的渐变，字体选择还不错。然后选项 B 上下都有一堆多余的东西。我选 A。Opus 做了 A。

英文原文

>> Okay. So, option A, I see like the green gradient. Decent font choices. And then option B has a bunch of extra stuff on top and bottom. I'm going to go with A. Opus did A.

你觉得 Opus 做了 A。好的。

英文原文

>> You think Opus did A. Okay.

是的。没错。

英文原文

>> Yes. Yes.

真的吗？

英文原文

>> Really?

你猜错了。Opus 做的是 B。而 A 实际上是 Grock 4.6 做的。

英文原文

>> You're wrong. Opus did B. And uh the A was actually Grock 4.6.

Grock。它的字体选择挺有意思，而且没有……对。好吧。

英文原文

>> Grock. It's got some interesting uh font choices and it doesn't Yeah. All right.

你让它生成一个按钮，结果它在按钮下方给了你三句类似标语的东西。所以是的，像那种……我对 open……

英文原文

>> You asked it for a button and it gave you three like taglines below it. So yes, like that's I don't have that experience with open

这个也有点类似。那么这里面哪一个是 Sonnet 45 做的？

英文原文

>> and this one is kind of similar. So which of these is sonnet 45?

好的。Sonnet 45。这是一年前、一年前的模型了。

英文原文

>> Okay. Sonnet 45. This is a year old a year ago.

是的。提示词同样要求生成一个按钮。所以［笑声］呃……

英文原文

>> Yes. Again the the prompt was asking for a button. So [laughter] uh

我是说，我这次还是要选 B，因为我想 Sonnet 那时候难道不是很不错吗？

英文原文

>> I mean again I'm gonna go with B because I think Sonnet wasn't Sonnet that decent back then?

Sonnet 45。是的。

英文原文

>> Sonnet 45. Yeah.

好的。我们来看看。CJ，你选 B。

英文原文

>> Okay. Let's see. CJ, you're going with B.

B。

英文原文

>> B.

好的。

英文原文

>> Okay.

我太差……噢不，我知道了。我太不擅长猜这个了。

英文原文

>> I am hor Oh, no. I know. I'm horrible at this.

不。Son of 45 实际上正是那个……好的。另一个是 Gemini 31，它做得相当不错，在这儿完成得挺出色的。

英文原文

>> No. Son of 45 was in fact the one that Okay. Uh, and the other one was Gemini 31, which did a pretty decent do pretty decent job here.

我才刚意识到这一点。嗯，我想我更喜欢其他模型。

英文原文

>> I'm just learning that. Um, I I like other models, I guess.

是的。对。是的。路过的朋友们，你们随时都可以加入。我们正在玩“猜猜这是哪个模型制作的”。这个的提示词也是一个按钮。我们要求生成一个按钮。这里面哪一个是 Muse Wonder 1.0 做的？

英文原文

>> Yes. Yes. Yes. Uh, folks, you can join at any time if you're walking by. We're playing Guess Which model made this. So, the prompt for this one uh was also a button. We asked this to make a button. Which of these did Muse Wonder 1.0 make?

好的。我根本没用过 Muse Wonder。它的名字里带有 Wonder。我选 B。

英文原文

>> Okay. I haven't used Muse Wonder at all. It's got the name Wonder in it. I'm going to go with B.

看起来像一个。它看起来很奇妙（wonderous）。有这个词吗？

英文原文

>> Looks one. It looks wonderous. Is that a word?

两个模型也都没太达到只做一个单按钮的提示词要求。是的，我们这里需要一个按钮。嗯，其中一个有

英文原文

>> Two models also kind of failed the uh the prompt of a single button. Yes. We need a button here. Um and one of them has

这个糟糕的、呃，溢出问题。所以，让我们……让我们……

英文原文

this awful uh overflow issue. So, let's let's

噢，是的。

英文原文

>> Oh, yeah.

是的。Muse。

英文原文

>> Yes. Muse.

不错。Muse 决定为这个做一整张卡片，而不是一个按钮。所以，

英文原文

>> Nice. Muse decided to make a whole ass card for this instead of a button. So,

辨别 AI 生成的设计

好的。看下面这个。这里的传单哪一张是 Opus 45 做的？

英文原文

>> all right. Next one here. Which one of these flyers did Opus 45 make?

又是 45。所以是 4，不是 55，45 到现在差不多有八九个月，或者十个月了。

英文原文

>> Four five again. So, four, not five5, four five, which is like eight or nine months old, 10 months old at this point.

这是 45。是的。

英文原文

>> This is four five. Yes.

嗯，我是说，我有点把真正像样的背景和字体选择跟较新的模型联系在一起。所以，我要说……但是这里有很多 emoji。两张里面都有 emoji。45。

英文原文

>> Um, I mean, I kind of associate like the really decent backgrounds and font choices with the newer ones. So, I'm going to say, but there's a lot of emoji. There's emojis in both of them. Four, five.

确实有 emoji。是的。

英文原文

>> There is emoji. Yeah,

这个很难。好的，我选……我选 B。噢，45 选 B。

英文原文

>> this is hard. Okay, I'm going to go I'm going to go B. Oh, four five to B.

你说是 45 是 B。好的，四五个月前。我不记得了。是的。嗯，A 当然是 55。所以，确实好上不少。嗯，这些 emoji 确实是一个时代的特征，对吧？它们非常代表那个时代。大家如果有想玩的，随时都可以加入。我们来看下一组。这里的传单哪一张是 Grock 4.6 做的？

英文原文

>> You're going to say four, five is B. Okay, four or five months ago. I don't remember. Yes. Um, the A was of course 55. So, quite a bit better. Um, the emojis are certainly a tell of an era, right? They're very a tell of an era. Folks, you can join at any time if you want to play. We got the next one here. Which one of these flyers did Grock 4.6 do?

好，我们刚才了解了一点 Grock 的特点。

英文原文

>> Okay, we learned a little bit about Grock earlier.

你那边还在加载吗？加载出来了。

英文原文

>> Are they still loading for you? They load.

已经加载出来了。

英文原文

>> They loaded.

好的，显示出来了。

英文原文

>> Okay, there we go.

呃，又是这样，这个上面有 emoji。我倾向于认为 Grock 想标榜自己的与众不同，所以他们不用 emoji。我选 A。Grock 46 是 A。

英文原文

>> Uh, again, so this one has emojis. I want to think like Grock sets themselves apart so they don't use emojis. I'm going to go with A. Grock 46 is a.

你说它是 A。是的。好的。这两张哪一张是 Grock 4.6 做的？答案确实是 A。是的。

英文原文

>> You're going to say it's a Yeah. Okay. Which one of these did Grock 4.6 make? And it was a Yes,

我答对了三道。对吧。

英文原文

>> I got three. Right.

是的。另一个是 Sonnet 45，从这里的二和二就能看出来。Sonnet 肯定特别喜欢那些 emoji。

英文原文

>> Yes. The other one was Sonnet 45, which you can tell from two and two here. Sonnet beloveing those emojis for sure.

那些 emoji。是的，现在看来这就是特征了。

英文原文

>> Those emojis. Yeah, it's it's a tell at this point.

就是那些 emoji。是的。

英文原文

>> It's those emojis. Yes.

好的。接下来是这两张传单。哪一张是 Gemini 3.1 做的？

英文原文

>> All right. The next one here is these two flyers. Which one of these did Gemini 3.1 do?

Gemini 31。那是最新……那是最新的一代。那是现代的 Gemini。我希望它是 B，因为 A 看起来太简陋了。我选 B。Gemini 做了 B。

英文原文

>> Gemini 31. That's the late That's the latest one. That's modern Gemini. Like I would I would hope it was B because A looks very basic. I'm gonna go with B. Gemini did B.

CJ 说是 B。投票正在进来，呃……是的，A 上有一堆 emoji。哪一张是 Gemini 31 做的？

英文原文

>> CJ is saying B. The votes are coming in and uh yeah, A's got a bunch of emojis. Which one did Gemini 31 do?

Gemini 用 emoji？我不知道。我不用 Gemini。天哪。好吧。

英文原文

>> Gemini use emojis. I don't know. I don't use Gemini. Oh my god. Okay.

显然 Gemini 用的全是这些 emoji。是的。遗憾的是，

英文原文

>> Apparently Gemini uses all these emojis. Yeah. Sadly,

争点气吧，Gemini。这可是前沿模型啊。

英文原文

>> get your stuff together, Gemini. This is a frontier model.

是啊。是啊。［笑声］不过呢，是的。Muse Wonder 实际做得确实比我们刚才看的上一张好了一点点。

英文原文

>> Yeah. Yeah. [laughter] Well, yeah. Muse Wonder actually did a little bit better of a job than the last one we saw here. though.

好的。是的。

英文原文

>> Okay. Yeah.

好的。接下来我们看几个收件箱界面。

英文原文

>> Okay. Next up, we got some inboxes.

噢。

英文原文

>> Ooh.

这几个稍微紧凑一些。比较难看清楚。这几个里面哪一个是 Gemini 31 做的？

英文原文

>> So, these ones are a little bit tighter. They're a little bit harder to see. Which one of these did Gemini 31 did?

肯定是 A。像 B 要么是 ChatGPT，要么是 Claude。右边的感觉像 ChatGPT 的风格，比如字体选择。所以，我选 A。Gemini 做了 A。

英文原文

>> It's got to be A. Like B has either chat GPT or Claude. Like it feels like chat GPT ism's on the right like the font choices. So, I'm going to go with A. Gemini did A.

好。你觉得 Gemini 做了 A，而确实是 Gemini 做了 A。Grock 46 做了 B。是的。那看起来肯定像是去年的 Opus 之类的。是的。是的。

英文原文

>> Okay. You think Gemini did A and Gemini did A? Grock 46 did B. Yes. And that'd be looking like that'd be looking like last year's Opus or something for sure. Yeah. Yeah.

你要是跟我说那是 Opus 45 之类的我也信。

英文原文

>> You could have told me that was Opus 45 or something.

好的，下一个。这两个哪一个是 Opus 55？

英文原文

>> All right, next one. Which one of these is Opus 55?

A 还是 B？

英文原文

>> A or B?

假设这是现代的 Opus，但同样，这两个都有大量的 emoji。

英文原文

>> Let's say this is modern Opus, but again, both of these have tons of emojis.

嗯，

英文原文

>> Um,

两个都有 emoji。是的，我们还以为 emoji 已经不用了。并没有。

英文原文

>> both of them have emojis. Yes, we thought the emojis were gone. No,

我……我想说 A。不对，我选 B。Opus 做了 B。

英文原文

>> I I want to say A. No, I'm gonna go B. Opus did B.

好的，各位，CJ 说 Opus 做了 B。

英文原文

>> Okay, CJ says that Opus did B, folks.

纯凭直觉。我不知道。

英文原文

>> Just a gut feeling. I don't know.

直觉。哪个是哪个？实际上，你觉得……甚至让我问，你觉得选项 A 是什么？

英文原文

>> It's a gut feeling. Which one is which? And actually, what do you think? Let me even What do you think option A is?

某款我从来没用过的模型。我就这么说吧。

英文原文

>> Some some model I've never used. I'll say that.

看来你两边都猜对了。Muse Wonder 做了选项 A，Opus 55 做了选项 B。所以那道题你答对了。

英文原文

>> Looks like you're right on both. So, Muse Wonder did option A and Opus 55 did option B. So, you were right on that one.

太酷了。

英文原文

>> Cool.

好的，来看看。各位，我们这里还有最后一个。这当中哪一个是 Sonnet 45？

英文原文

>> Okay, let's see. We got one more here for you folks. Which one of these was Sonnet 45?

同样，Sonnet 45 到现在已经过去了 10 个月。呃，呃，这很难说，因为两个都挺平淡的。右边有 emoji。我想着大概一年前 AI 都在大量使用 emoji 吧。我选 B。好的，你说选 B。你觉得呃选项 A 是什么？

英文原文

>> Again, Sonnet 45 is 10 months ago at this point. Um, uh, this is a tossup because they're both pretty bland. There's emojis on the right. I'm gonna go with like AI was using a ton of emojis like a year ago. I'm gonna go with B. Okay, you're gonna say B. What do you think uh option A is?

我可能要按你的话来较真了。

英文原文

>> I may hold you to that.

Gemini。

英文原文

>> Gemini.

Gemini。好的。那么，

英文原文

>> Gemini. Okay. So,

噢，

英文原文

>> oh,

选项 B 是 Sonnet 45，呃选项 A 是 Opus 45。

英文原文

>> option B was set 45 and uh option A was Opus 45.

所以这些……这些模型是同一时期推出的。

英文原文

>> So, these these models were out at the same time.

是的。

英文原文

>> Yes.

然后，我是说，左边那个就很平淡，像……

英文原文

>> And then I mean the the one on the left was like very bland like

是的。

英文原文

>> Yes.

我猜是没什么特色吧。

英文原文

>> No character, I guess.

我是说，一个电子邮件收件箱需要多少特色？我想我还是需要点特色的。叫什么来着？Google Inbox 是我一直以来最喜欢的邮件应用。

英文原文

>> I mean, how much character do you need in an email box? I guess I'd need some character. What was it? Google Inbox was my favorite email app of all time.

不错。

英文原文

>> Nice.

很酷。行吧。“猜猜这是哪个大语言模型”就到这里。我们回到正题。我们这里还有一些观众提问。下一道问题是：既然现在没人把 Google 当搜索词用了，“Google”这个词会不再作为动词使用吗？没人在说“让我 Google 一下”了。如果搜索不复存在，Google 要怎么维持业务？听着，Google……就像我们已经……现在大多数用 Google 的人都在看 AI 的答案，对吧？也就是 AI Overview。所以，呃，我认为那个词仍然存在。就像那个动词仍然在那里。比如“让我 Google 一下”，然后他们根本不看结果，只看 AI 的答案。

英文原文

>> Sick. All right. Well, that's it for which LLM was this. Let's get back. We also had some audience questions in here. So, next question was going to be now that no one uses Google as a search term anymore. Will the word Google stop being a verb? No one is saying let me Google that. If search isn't a thing anymore, how is Google going to stay in business? Listen, Google like we already like most people using Google now use the AI answers, right? The AI overviews. So, um I think that term is still there. Like that verb is still there. Like let me Google it and then they don't look at the results. They just look at the AI answer.

是的。

英文原文

>> Yes.

嗯，我用……我依然会说 Google，而我都不用 Google。我用 DuckDuckGo。所以可能要过很长一段时间我才能摆脱把它当成动词。我不知道，你呢？

英文原文

>> Um I use I still say Google and I don't use Google. I use duck.go. So like it's going to be a while before I shake it as a verb. I don't know. What about you?

是啊。难道要花上一整代人的时间，“Google”这个词才会彻底消失吗？到时候大家会……就像那个 iOS 电脑广告一样，比如“什么是电脑”，变成“什么是 Google”。对啊，我……我是说我不知道，我本来就没怎么把 Google 当动词用，所以我认为……我认为这大多已经差不多结束了。

英文原文

>> Yeah. What is it going to take a whole generation before Google is gone as a term? where people are going to be it's going to be that that I iOS computer that iOS commercial like what's a computer like what's a Google yeah I I mean I don't know I I don't really say Google anyways as a verb so I think I think it's mostly done

而且我听过有人说“让我 GPT 一下”，所以那成了新的动词。

英文原文

>> and I've heard people say like let me GPT that so that's like the new verb

是的，确实……我确实在日常生活里听到过这种说法，比如在机场之类的场合，而且还不是加州的机场，是在丹佛的……是的，

英文原文

>> yeah people people actually I have been hearing that in the wild like at the airport and stuff like that and not in California's airport in uh in Denver's Yeah,

应该还有时间回答最后一个问题。

英文原文

>> probably got time for one more.

好的。我们这里还有最后一个问题。呃，你想挑哪一个，CJ？

英文原文

>> Yeah. Okay. So, we got one more one more question in here. Uh, which one do you want to pick, CJ?

你有特别想选的问题吗？我们有一个关于 WebMCP 的，还有一个关于呃……

英文原文

>> You got a question you want to pick in particular? We got one about WebMCP. We got about uh

在 AI 时代开发应用的意义

我们选关于 App 的那个吧，因为这个……这个非常切合实际，比如对现在任何想做点东西的人来说。是的。所以，为什么还要发布 App 呢？对。

英文原文

>> let's do the let's do the one about apps because this is this is very relevant like anybody trying to build something right now. Yes. So, why release an app? Yeah.

好的。进来的这个问题写道：“以前我们会为自己做一些 App，然后发布给全世界，希望能对某些人有用，进而或许能树立一些声誉、赚点小钱。但现如今为什么还要发布 App 呢？要是它做得好，一周内就会冒出 100 个粗制滥造的仿制品，然后全都淹没在炮火中了。”

英文原文

>> Okay. So this question came in and it says, "We used to build apps for ourselves and then release them to the world in the hope that someone may find them useful and then maybe develop a reputation and make a few dollars. Why release an app nowadays though when if it's any good there's going to be 100 slop cones within a week and it'll all get lost in the fire?"

是的。

英文原文

>> Yeah.

是啊。我最初的想法是，如果你真正倾注了心血和思考，而不是像随便用 vibe coding 写写，然后扔一个 GitHub 仓库链接或者别的什么……人们是会在意的。当大家成为一个 App 的真正用户，并且……呃，还会再回来用它，我认为是因为他们知道这里面倾注了心血，对吧？比如……比如良好的用户体验。嗯，或者像我现在正在做的一款开发者工具。它跟这个不完全一样，但我花了很多精力在文档上，花了很大力气确保人们能明白这个工具是什么，

英文原文

>> Yeah. My initial thoughts are like if you actually put care and thought into it and don't just like vibe code and then throw up a link to a GitHub repo or whatever else like people are going to care like the when people become a like actual user of an app and uh come back to it. I think it's because they know that there was care put into it, right? Like like it's a nice user experience. Um or like so right now I'm working on a developer tool. It's not exactly like this, but I put so much effort into the docs and so much effort into making sure that people understood what this tool was,

倾注关爱。

英文原文

>> some TLC

以及它能做什么。而且像……

英文原文

>> and what it could do. And like

呃，很多人就是……他们把工具做出来，然后让 AI 去写 Readme，自己完全不花任何心思去想。我觉得这当中依然有生存空间，但你必须用心。是的。

英文原文

>> uh a lot of people are just like they build the tool and then like let AI write the read me and don't put any thought into I think there's still space for it, but you have to care. Yeah.

因为否则你就会成为大家粗制滥造拼凑出的那片 App 汪洋里的又一款普通应用而已。所以我觉得归根结底就是用心。

英文原文

>> Like because you'll just be one more app in the sea of apps that everybody just slopped together. So I think care is what it comes down to.

是的。归根结底就是用心。我认为关于这一点的完美范例……

英文原文

>> Yeah. Care is what it come down to. I think the the perfect example of this

是的。

英文原文

>> Yeah.

就是 Rex。Rex 终端。呃，天哪，

英文原文

>> is Rex. The Rex terminal. Um my gosh,

Rex 终端。

英文原文

>> the Rex terminal.

是 Hex 还是 Rex？

英文原文

>> Hex or Rex?

Rex。

英文原文

>> Rex.

Rex。好的。

英文原文

>> Rex. Okay.

Rex。是的。Rex 是终端。Hex 是呃语音……

英文原文

>> Rex. Yeah. Rex is the terminal. Hex is the uh voice

语音。

英文原文

>> voice.

然后 Dex 是问题追踪器。天哪。好的。所以，Rex 是一个倾注了大量心血打造的应用。你能看出来它被倾注了大量心血，呃，因为一切……哪怕是提示有更新的小通知都做得非常精美。它……它就像是有跑马灯效果。设计得很出色

英文原文

>> And then Dex is the issue tracker. My gosh. Okay. So, Rex is an app that's built with a lot of care. And you can tell it's built with a lot of care uh because everything even the little notification that says you have an update is beautiful. It's it's like got a marquee. It's designed well

就好像这个应用程序的每一个细微角落都倾注了心血与巧思。嗯，当你有了这样的作品，并把它与粗制滥造的替代品进行对比时，就会非常清楚地看到，那些用心投入的人依然会赢得关注，依然会获得用户，也依然会收获下载量。如今有些人靠开发小型开发者工具赚了不少钱，他们把工具发布给欣赏自己作品的受众群体，这些人依然乐意为此买单，

英文原文

and

it

is

like

thought

and

care

is

put

into

every

tiny

corner

of

this

application.

Um,

and

when

you

have

things

like

that

and

you

compare

it

to

a

slopped

alternative,

it

makes

it

very

clear

that

the

people

who

are

putting

the

care

into

it

are

still

going

to

get

uh

attention.

They're

still

going

to

get

users

and

they're

still

going

to

get

downloads.

There's

people

who

are

making

a

lot

of

money

building

small

developer

tools

these

days,

releasing

them

to

an

audience

of

people

who

like

their

work

and

then

still

happily

pay

for

that

stuff,

对吧？所以，你确实会听到所有人都在说，他们如今都在 vibe coding 自己的个人软件，我也是如此。相信我，如今我自己也用 vibe coding 编写了各种各样的个人软件，但大浪淘沙，优胜劣汰，出类拔萃的作品依然会脱颖而出。所以，

英文原文

>> right?

So,

you

do

hear

everybody

saying

they're

they're

all

vibe

coding

their

own

personal

software

these

days

and

I

am

too.

Trust

me,

there's

all

kinds

of

stuff

I'm

vibe

coding

as

personal

software

these

days

as

well,

but

the

cream

still

rises

to

the

top.

So,

我百分之百赞同。

英文原文

>> I

agree

100%.

好的。CJ，我们这边时间到了吗？

英文原文

>> Yes,

CJ,

what

are

we

at

for

time

here?

呃，我是说，我认为严格来说我们已经到时限了。所以，

英文原文

>> Uh,

I

mean,

I

think

technically

we're

at

our

limit.

So,

好的，各位，以上就是我们在 AIE 的全部内容了。我们现在在旧金山。这里的会议将会非常精彩。我的名字是 Scott Tinsky，这位是 CJ Reynolds。更糟糕的语法。再见。

英文原文

>> okay,

folks,

this

has

been

it

from

AIE.

We're

in

San

Francisco.

It's

going

to

be

a

great

conference

here.

My

name

is

Scott

Tinsky.

This

is

CJ

Reynolds.

and

worse

syntax.

Peace.

数据流水线与 AI 评审

想象一个 Pull Request。它修改了我们其中一条数据流水线中的模式定义（schema）。它在 CI 中显示为绿色，测试全部通过，linter 检查合格，关于它的一切看起来都很干净。

英文原文

Picture

a

pull

request.

It

changes

a

schema

in

one

of

our

data

pipelines.

It

goes

green

in

CI.

The

tests

pass.

The

llinter

is

happy.

And

everything

about

it

looks

clean.

那么，合并它是否安全？因为像这样的更改仍然可能破坏下游的某些东西。也许它破坏了新鲜度 SLA，或者也许破坏了隔了三个团队、对此毫不知情的下游消费者。又或者也许它暴露了本不该被访问的数据。CI 通过与这个变更实际上安全之间的差距，正是我的团队每天必须去弥合的鸿沟。大家好，我是 Jas。我是 Robinhood 数据平台团队的一名工程师。我设计并构建了我们数据湖背后的分布式数据摄入和处理框架，全公司的各个团队都依赖这些框架来利用数据做出决策。

英文原文

So,

is

it

safe

to

merge?

Because

a

change

like

this

can

still

break

something

downstream.

Maybe

it

blows

a

freshness

SLA

or

maybe

it

breaks

a

consumer

three

teams

away

who

never

saw

it

coming.

Or

maybe

it

exposes

data

that

never

supposed

to

be

reachable.

That

gap

between

CI

past

and

this

change

was

actually

safe

is

the

gap

my

team

has

to

close

every

day.

Hello

everyone,

I'm

Jas.

I'm

an

engineer

on

the

data

platform

team

at

Robin

Hood.

I

design

and

build

the

distributed

data

injection

and

processing

frameworks

behind

our

data

lake.

The

ones

teams

across

the

company

rely

on

to

make

decisions

with

data.

请注意这次演讲的标题。其中的 Bot 既指那些提交 Pull Request 的 Bot，正如你们将看到的，也是进行代码评审的 Bot。

英文原文

Notice

the

title

of

this

talk.

The

bots

in

it

are

the

ones

opening

the

pull

requests

and

as

you

will

see

the

ones

reviewing

them

as

well.

今天我想向大家展示，当 PR 涌入的速度超过我们任何人阅读的速度时，我们是如何回答那个问题的。

英文原文

Today

I

want

to

show

you

how

we

answer

that

question

when

the

PRs

are

coming

in

faster

than

any

of

us

can

read

them.

用一句话概括这个问题就是：针对我们生产数据流水线的 Pull Request 如今正以机器速度涌入，但安全地评审它们却仍然以人类的速度运行。

英文原文

So

here's

the

problem

in

one

sentence.

Pull

requests

to

our

production

data

pipelines

now

arrive

at

machine

speed

but

reviewing

them

safely

still

runs

at

human

speed.

首先简要说明一下本次演讲的范围，因为它决定了这次演讲中的一切内容。

英文原文

First

a

quick

word

on

the

scope

of

this

talk

because

it

shapes

everything

else

in

this

talk.

AI 编程助手让提交 Pull Request 变得成本极低。一个新的查询、一次 schema 变更或一次权限授予，现在只需要几分钟。而且助手提交 Pull Request 的速度，远远快于任何人类团队所能达到的速度。

英文原文

AI

coding

assistants

have

made

pull

requests

cheap

to

open.

A

new

query,

a

schema

change,

or

an

access

grant

now

takes

just

a

few

minutes.

And

an

assistant

can

open

pull

requests

far

faster

than

any

team

of

humans

ever

could.

我今天谈论的 PR，是指针对我们数据流水线的 PR，比如 schema 变更、流水线代码、配置变更，以及授予某人数据访问权限的授权。

英文原文

The

PRs

I'm

talking

about

today

are

the

PRs

to

our

data

pipelines,

like

schema

changes,

pipeline

code,

configuration

changes,

and

grants

that

give

someone

access

to

data.

数据平台是负责评审这类 PR 的团队。因此，当这些 PR 积压起来时，就全堆积在了我们身上。而我即将描述的评审器，正是我们为了处理它们而构建的一个系统。因此，感受到这种痛点的是一小部分人：掌握着能够判断合并到生产环境中是否安全这一上下文的工程师，以及所有等待他们的人。

英文原文

data

platform

is

the

team

that

owns

reviewing

that

kind

of

PR.

So

when

those

PRs

pile

up,

they

pile

up

on

us.

And

the

reviewer

I'm

about

to

describe

is

a

system

we

built

to

handle

them.

So

the

people

who

feel

this

are

a

small

group.

The

engineers

who

carry

the

context

to

say

what

is

safe

to

merge

into

production

and

everyone

who

is

waiting

on

them.

那么为什么这是个问题呢？由 AI 提交的 PR 源源不断地到来。平心而论，我们已有的安全网是有效的。CI 检查与人工评审相结合，一直以来都能发现许多真正的错误，比如损坏的代码、低效的查询，以及诸如下游损坏或敏感数据泄露等风险。但是人类这一半只能以人类的速度运作，并且只能依靠某个人恰好印在脑子里的上下文。谁在读取这张表？它有多大？里面的数据有多敏感？PR 正在以机器速度涌入，但是理解合并到生产数据流水线中是否安全的人员数量，却根本没有增加。这就是不匹配之处。队列在少数拥有上下文的人面前积压起来，问题正是从这里开始滑坡。

英文原文

So

why

is

this

a

problem?

AI

open

PRs

just

keep

arriving.

And

to

be

fair,

the

safety

net

we

already

had

works.

CI

checks

together

with

the

human

review

have

always

caught

a

lot

of

real

bugs

like

broken

code,

inefficient

queries

and

risks

like

downstream

breakage

or

sensitive

data

leaks.

But

the

human

half

only

works

at

the

human

speed

and

only

with

the

context

a

person

happens

to

carry

in

their

head.

Who

reads

this

table?

How

big

is

it?

And

how

sensitive

is

the

data

in

it?

The

PRs

are

coming

in

at

the

machine

speed,

but

the

number

of

people

who

understand

what

is

safe

to

merge

into

a

production

data

pipeline

did

not

grow

at

all.

That's

the

mismatch.

The

queue

builds

up

in

front

of

the

few

people

with

the

context

and

that's

where

things

start

to

slip.

因此，我们着手要解决的问题就是：利用 AI 构建一个 PR 评审器，能够跟上 PR 提交的速度。

英文原文

So

the

problem

we

set

out

to

solve

was

this.

Build

a

PR

reviewer

using

AI

that

can

keep

up

with

the

speed

at

which

PRs

are

being

opened.

在深入细节之前，这里是完整的答案。三项职责，三方归属。脚本负责验证能够被验证的事实，Agent 负责研判需要上下文的事情，而人类则掌控唯有他们才应做出的决策。我们采取了混合方案：脚本负责事实，AI 负责裁决。我们不希望 AI 去评审每一项变更，因为某些问题已经具备权威的根据……享用了午餐，正在布置并关闭这个房间。退出键注册表或目录……

英文原文

So

here's

the

whole

answer

before

we

get

into

the

details.

Three

duties,

three

owners.

Scripts

prove

what

can

be

proven.

An

agent

judges

what

needs

context.

And

people

own

the

calls

only

they

should

make.

We

took

a

hybrid

approach.

Scripts

for

the

facts

and

AI

for

judgment.

We

did

not

want

AI

reviewing

every

change

when

some

questions

already

have

an

authority

have

enjoyed

lunch

and

that's

setting

and

closing

this

room.

Exit

the

key

registry

or

a

catalog

门将重新……

英文原文

>> doors

will

reop

并在修复之前阻止该 PR。

英文原文

and

blocks

the

PR

until

it

is

fixed.

第二层是 Agent。该 Agent 的规则会告诉它第 1 层已经检查了什么，因此它专注于那些检查无法捕获的风险。它评审 PR，评估风险评分，并予以批准或上报给人工处理。这两层都会评审同一个 PR。合并需要既通过各项检查，又获得批准性评审。

英文原文

>> Layer

two

is

the

agent.

The

agents

rules

tell

it

what

layer

1

already

checks.

So

it

focuses

on

the

risks

those

checks

cannot

catch.

It

reviews

the

PR,

scores

the

risk

and

approves

or

escalates

to

a

person.

Both

these

layers

review

the

same

PR.

Merge

requires

passing

checks

and

an

approving

review.

这也降低了 AI 的成本。我们用 AI 来做裁决，而不是去重新发现脚本就能查到的事实。

英文原文

And

this

also

keeps

AI

costs

down.

We

use

AI

for

judgment,

not

to

rediscover

facts

a

script

can

look

up.

这就是我们今天的内容走向。我们已经讨论了问题和应对方法。接下来，我将向大家详细介绍我们构建的混合评审器，以及我们特意保留由人工负责的那些决策。之后，介绍该评审器如何从人工反馈中学习，以及我们目前为止看到了哪些成果。最后，我们将以想分享给大家的经验教训来收尾。

英文原文

So

here

is

where

we

are

going

today.

We

have

covered

the

problem

and

the

approach.

Now,

I'll

walk

you

through

the

hybrid

reviewer

we

build

and

the

decisions

we

deliberately

keep

with

the

people.

After

that,

how

the

reviewer

learns

from

the

human

feedback

and

what

results

we

have

seen

so

far.

And

we'll

finish

with

the

lessons

we

would

pass

on

to

you.

在我们进行的过程中，请关注三件事：用于事实的脚本，用于裁决的 Agent，以及用于受保护决策的人类。在最后我会再次回顾这三点。

英文原文

Three

things

to

listen

for

as

we

go.

Scripts

for

facts,

an

agent

for

the

judgment,

and

people

for

the

protected

cause.

I'll

come

back

to

all

three

at

the

end.

为了使内容具体化，让我向大家展示一个在接下来演讲中我们将全程跟踪的 PR。

英文原文

So

to

make

this

concrete,

let

me

show

you

the

one

PR

will

follow

through

the

rest

of

this

talk.

这个 PR 对我们数据湖中的一张表做了两处更改。它添加了一个新列，并将一个现有列从 integer 拓宽为 big integer。这是我们经常见到的那种 PR。左边是一个上游源表。我们的摄入流水线从它那里读取数据，并写入数据湖中的一张表——姑且称之为我们的表。

英文原文

This

PR

makes

two

changes

to

a

table

in

our

data

lake.

It

adds

a

new

column

and

widens

an

existing

column

from

integer

to

big

integer.

It's

the

kind

of

PR

we

see

all

the

time.

On

the

left

is

an

upstream

source

table.

Our

injection

pipeline

reads

from

it

and

writes

into

a

table

in

the

data

lake

call

it

our

table.

第一处更改：源表增加了一个新列，而这个 PR 让摄入程序把这个新列引入我们湖中的表中。

英文原文

The

first

change

the

source

has

gained

a

new

column

and

this

PR

makes

injest

bring

that

new

column

into

our

lake

table.

第二处更改是针对该表已有的一个列，其类型从 integer 拓宽为 big integer，以便能够容纳更大的数值。这听起来本身完全无害。拓宽类型几乎是你所能做的最安全的 schema 变更了，对吧？

英文原文

The

second

change

is

to

a

column

of

a

table

already

has

its

type

gets

widened

from

integer

to

big

integer

so

it

can

hold

bigger

values

on

its

own.

That

sounds

completely

harmless.

Widening

a

type

is

about

the

safest

schema

change

you

can

make,

right?

然而，每当一个列发生变化时，真正的问题是下游谁在读取它？其中一些读取者在我们的血缘工具中注册过，所以我们知晓他们。

英文原文

Well,

whenever

a

column

changes,

the

real

question

is

who

reads

it

downstream?

Some

of

those

readers

are

registered

in

our

lineage

tool.

So,

we

know

about

them.

但请在脑海深处保留第二个问题：他们就是全部的读取者吗？

英文原文

But

keep

a

second

question

in

the

back

of

your

mind.

Are

those

all

of

them?

请记住这张图。每当这个 PR 在评审器中推进时，我们都会回到这张完全相同的图表。

英文原文

Hold

on

to

this

picture.

We

will

come

back

to

this

exact

diagram

each

time

the

PR

moves

forward

through

the

reviewer.

那么让我们从第一层开始。第一层是脚本，它们的全部职责就是在生产运行系统中检查事实——这是仅凭代码无法告诉你的事情。

英文原文

So

let's

start

with

layer

one.

Layer

one

is

the

scripts

and

their

whole

job

is

checking

facts

in

the

live

systems

things

the

code

alone

cannot

tell

you.

这里是其中两项检查，以及它们对我们当前这个 PR 的检查结论。

英文原文

Here

are

the

two

of

those

checks

and

what

they

said

about

our

running

PR.

首先是数据契约。当一个 PR 更改模式时，脚本会将该模式和键与权威数据源——即 schema registry（模式注册表）进行比对。对于我们的 PR 来说，问题很简单：那个新列是否确实像注册表中注册的那样存在于上游？确实存在。因此这项检查通过了。

英文原文

First

data

contracts.

When

a

PR

changes

a

schema,

a

script

compares

the

schema

and

keys

against

a

source

of

truth,

the

schema

registry.

For

our

PR,

the

question

is

simple.

Does

that

new

column

actually

exist

upstream

as

registered

in

the

registry?

It

does.

So

that

check

passes.

其次是必需标签。每张表都需要有一个所有者，并且任何 PII（个人身份信息）都必须在目录中打上标签。脚本会直接检查目录：我们的表有所有者吗？有。它的 PII 打上标签了吗？打了。再次通过。

英文原文

Second,

require

tags.

Every

table

needs

an

owner

and

any

PII

has

to

be

labeled

in

the

catalog.

The

script

checks

the

catalog

directly.

Does

our

table

have

an

owner?

Yes.

Is

its

PII

labeled?

Yes.

Pass

again.

如果这两项中有任何一项失败，没有任何商量的余地，也不需要模型介入。CI 会像遇到其他任何失败检查一样变红，并且该 PR 会被阻止，直到被修复为止。契约和标签属于第 1 层的其中两项检查。接下来是审视下游影响和性能的两项检查。因此，这就是第 1 层的另外两项检查。它们超越表本身去审视：下游会发生什么，以及任务是否仍能按时运行。

英文原文

And

if

either

of

those

had

failed,

there

is

no

debate

and

no

model

involved.

CI

goes

red

like

any

other

failing

checks

and

the

PR

is

blocked

until

it

is

fixed.

Contracts

and

tags

are

two

of

the

layer

1

checks.

Next,

the

two

that

look

at

downstream

impact

and

performance.

So,

these

are

the

other

two

layer

one

checks.

They

look

past

the

table

itself,

what

happens

downstream

and

whether

the

job

will

still

run

in

time.

在我们进入闭幕主题演讲之际，我们非常高兴能请回大家熟悉的一位……

英文原文

>> As

we

head

into

our

closing

keynotes,

we're

excited

to

bring

back

a

familiar

to

the

one.

女士们先生们，请热烈欢迎我们的主持人 Corey Quinn，The DuckBill Group 的首席云经济学家。它们中是否有任何一个被破坏？这是针对真实系统的真实检查。但请注意我画出的边界。它只能看到血缘中记录的内容。它对下游的观察视角，完完全全只与那份记录一样完整，仅此而已。第二，性能。一个正确的变更仍然可能……

英文原文

>> Ladies

and

gentlemen,

please

welcome

back

our

MC

Cory

Quinn,

chief

cloud

economist

at

the

Duck

Bill

Group.

Does

any

of

them

break?

It's

a

real

check

against

a

real

system.

But

notice

the

boundary

I

have

drawn.

It

can

only

see

what

is

recorded

in

lineage.

Its

view

of

downstream

is

exactly

as

complete

as

that

record.

No

more.

Second,

the

performance.

A

correct

change

can

still

太慢或太昂贵。因此，CI 性能测试使用模拟数据，在暂存环境中以与生产环境相同的数据量运行该变更。[清嗓子] 这在 PR 合并之前告诉我们两件事：集群规模是否适合该作业，以及该作业是否仍能满足其新鲜度 SLA。那些正是否则会在几周后作为故障显现出来的问题。所以这就是第一层检查的完整集合。让我们看看它们对我们的 PR 怎么说。

英文原文

be too slow or too expensive. So the CI performance test runs the change in staging at the same data volume as [clears throat] production using mock data. So that tells us two things before the PR merges. Whether the cluster is sized right for the job and whether the job will still make its freshness SLA. Those are the kind of problems that otherwise show up weeks later as an incident. So that's the full set of layer 1 checks. Let's see what they said about our PR.

回到我们正在运行的 PR。和之前一样的画面。以下是第一层对其的说明。

英文原文

So, back to our running PR. Same picture as before. Here's what the layer one said about it.

数据契约检查访问了模式注册表，询问新列是否确实像注册的那样存在于上游。它确实存在，检查通过了。

英文原文

The data contract check went to the schema registry and asked whether the new column really exists upstream as registered. It does and the check passes.

血缘爆炸半径检查遍历了加宽列的每一个已注册下游消费方。血缘工具记录在案的作业。当该列从整数变为大整数时，它们都没有损坏。它们看起来都很好。

英文原文

The leanage blast radius check walked every registered consumer of the widen column. The jobs the lineage tool has on record. None of them break when the column goes from integer to big integer. They all look fine.

随之，标签通过，性能通过，每一项检查都是绿色的。

英文原文

Along with that tags pass, performance passes, every check is green.

就脚本所能看到的而言，这个 PR 是安全的。但我仔细推敲了这些词句。就脚本所能看到的而言，幻灯片上的所有内容都是关于已注册画面的陈述。我们的系统记录在案的作业和列。请记住这一点。这并不是全貌。

英文原文

As far as the scripts can see, this PR is safe. But I chose those words carefully. As far as the scripts can see, everything on the slide is a statement about the registered picture. the jobs and the columns our systems have a record of. And keep that in mind. It is not the whole story.

代码审查 Agent 与代码遍历

现在，让我们看看带有 diff 的 Agent。究竟改变了什么？

英文原文

Now, let's look at the agent with the diff. What actually changed?

PR 描述和提交信息解释了作者的意图，但意图绝不能开脱真正的发现。如果描述说它是安全的，而代码显示并非如此，以代码为准。

英文原文

The PR description and the commit messages explain the author's intent, but intent never explains away a real finding. If the description says it is safe and the code says otherwise, the code wins.

文件工具允许 Agent 读取、搜索和列出整个代码仓库中的文件。它追踪 diff 之外的变更，深入到流水线定义和访问授权中，寻找作者可能忽略的关系。

英文原文

File tools let the agent read, search and list the files across the repository. It follows changes beyond the diff into pipeline definitions and access grants looking for the relationships the author may have missed.

这些仅是代码仓库工具。Agent 无法查询实时元数据目录、血缘工具或生产状态。实时事实仍然是第一层的工作。Agent 对代码进行推理。那么为什么要针对模式变更去遍历那些代码呢？因为已注册的血缘并不是全貌。

英文原文

These are the repository repository tools only. The agent cannot query the live catalog, lineage tool or production state. Life facts stay layer 1's job. The agent reasons over the code. So why walk that code for a schema change? Because registered lineage is not the whole picture.

规则定义了那种判断的归属。它们根据 PR 中的变更类型进行配置，已知第一层已经涵盖的内容。它们针对查询反模式、敏感数据索取以及血缘之外的消费方。

英文原文

The rules define where that judgment belongs. They are configured by the change type in the PR knowing what layer 1 already covers. They target query antiatterns, sensitive data asks and consumers outside lineage.

例如，一个未分区的交叉连接可以正确运行，但仍然会冲击集群。一个授权看起来可能很寻常，但仍然会开放敏感数据。

英文原文

For example, an unpartitioned cross join can run correctly and still hammer a cluster. A grant can look ordinary and still open sensitive data.

对于模式变更，Agent 遍历代码仓库中的流水线定义和访问授权，以查找未注册的消费方。这是一项繁琐的搜索，嗯，否则需要人工手动完成。

英文原文

And for the schema changes, the agent walks the pipeline definitions and access grants in the repository to find consumers that were not registered. It's a tedious search uh a person would otherwise do by hand.

让我们回到我们的 PR，看看那项搜索发现了什么。

英文原文

Let's return to our PR and see what that search found.

回到我们正在运行的 PR。同样的画面。第一层表示加宽列的每个已注册消费方看起来都很好。现在 Agent 遍历代码，

英文原文

So back to our running PR. Same picture. Layer one said every registered consumer of the vid column looks fine. Now the agent walks the code

它发现了一个血缘从未知道的作业。它就在代码仓库里。它从我们的宽表中读取我们加宽的列，但它从未在血缘中注册过，这正是第一层检查无法看到它的原因。

英文原文

and it finds a job that lineage never knew about. It's right there in the repository. It reads our widened column from our late table but it was never registered in the lineage which is exactly why the layer 1 check could not see it.

该作业获取该值并将其写入另一个表。而在那另一个表中，该列仍然是整数。

英文原文

And that job takes the value and writes it into another table. And in that other table, the column is still an integer.

现在顺着推演下去。我们的列是一个大整数。现在，第一次传来一个大于整数所能容纳的值时，该写入就会失败。一个有效的模式链，在 CI 中全绿，通过了每一项已注册的检查，但下游作业仍然中断。

英文原文

Now follow it through. Our column is a big integer. Now the first time a value comes through that's bigger than an integer can hold, that write fails. A valid schema chain green in CI passes every registered check and a downstream job still breaks.

Agent 在其审查评论中准确标记了这一链路并附带了推理依据。这样人就可以看到从我们的列到失败写入的完整路径。我们稍后会讲到接下来发生了什么。

英文原文

The agent flagged exactly this chain in its review comment with the reasoning. So a person could see the whole path from our column to the failing right. We'll get to what happened next in a moment.

那么为什么血缘会漏掉那个作业呢？因为企业数据世界是混乱的。工作负载运行在不同的查询引擎上，每个引擎都必须发布自己的血缘。如果有一个不发布，该关系就会从画面中缺失。并不存在能自动看到一切的单一目录。所以已注册的画面是有用的，但它并不是全貌。

英文原文

So why did the lineage miss that job? Because the enterprise data world is messy. Workloads run across different query engines and each engine has to publish its own lineage. If one does not publish, that relationship is missing from the picture. There's a single catalog that automatically sees everything. So the registered picture is useful but it is not the whole picture.

现在我不想过分夸大代码遍历。它确实有用，但它无法看到一切。

英文原文

Now I don't want to oversell the codewalk. It's genuinely useful but it cannot see everything.

Agent 可以搜索代码仓库中的内容。流水线代码、访问授权，以及其中定义的读取我们表的作业。那就是它找到我们未注册作业的地方。但大量的数据使用存在于代码库之外。有人手动运行的即席查询。

英文原文

The agent can search what is in the repo. Pipeline code access grants and the jobs defined there that read our tables. That's where it found our unregistered job. But plenty of data use lives outside the code base. Ad hawk queries someone runs by hand.

仪表板，这些对于代码仓库搜索来说是完全隐形的。无论 Agent 多么仔细地寻找，外面的消费方永远不会显现。所以我们让 Agent 诚实面对这种限制。如果它没有找到任何东西，它不能说不存在下游消费方。它会说明在它搜索过的地方没有找到。没有找到并不等同于不存在。而且 Agent 永远不能宣称彻底排除隐患。

英文原文

Dashboards, those are completely invisible to a repository search. A consumer out there never shows up no matter how carefully the agent looks. So we make the agent honest about that limit. If it does not find anything, it does not get to say no downstream consumers exist. It says it found none in the places it searched. None found is not the same as none exist. And the agent never gets to claim an all clear.

审查产出与人工协同

那么 Agent 遍历了代码。它实际上产出什么？一份审查。就像优秀的真人审阅者会给你的形式一样。

英文原文

So the agent has walked the code. What does it actually produce? A review. The same shape a good human reviewer would give you.

首先，针对关键具体行的审查评论。每一条都带有其推理依据。不是含糊的“看起来有风险”，而是“这一行修改了这一列，这里是读取它的作业”。

英文原文

First review comments on the specific lines that matter. Each with its reasoning. Not a vague looks risky, but this line changes this column. And here is the job that reads it.

其次，风险评分。衡量它所发现问题的严重程度。正是这一点驱动了下一步。

英文原文

Second, a risk score. How severe is what it found. That's what drives the next step.

然后它做出裁决。如果没有严重风险，它可以批准。如果存在需要人介入的风险，它会升级给人工，评论和推理依据也一并转交。这样人就有了上下文去调查，而不必从头开始。

英文原文

Then it makes a call. If there is no serious risk, it can approve. If there is a risk that needs a person, it escalates to a human and the comments and reasoning go with it. So the person has the context to investigate instead of starting from scratch.

明确说明合并需要满足的条件：第一层的检查必须通过，且审查必须批准——要么由 Agent 批准，要么在升级后由人工批准。双方都必须同意。现在让我们看看我们的 PR 是怎样的情况。

英文原文

And to be clear about what it takes to merge, layer 1's checks have to pass and the review has to approve either the agent or after an escalation a person. Both have to say yes. Now let's see what that looked like for our PR.

这就是我们正在运行的 PR 实际的结局。同样的画面。已注册的消费方看起来没问题，而代码遍历发现了一个未注册的作业，该作业将我们加宽的列写入一个其类型仍为整数的表中

英文原文

Here is how our running PR actually ended. Same picture. The registered consumers look fine and the core walk found an unregistered job that writes our widened column into a table where it is still an integer

因为 Agent 发现了一个真正的下游故障。它评定风险为高，并升级给人工，在审查评论中阐明了整个链路。现在由人工来完成 Agent 无法完成的部分。Agent 仅对代码仓库进行推理。它无法看到该作业是否实际上正在生产环境中运行。

英文原文

because the agent found a real downstream break. It scored the risk high and escalated to a human with the whole chain spelled out in its review comment. Now the human does the part the agent cannot. The agent only reasons over the repo. It cannot see whether the job is actually running in production.

人工审阅者进行了检查，并确认 Agent 发现的消费方是实时且活跃的。所以这不是一个理论上的风险。PR 进行了等待

英文原文

human reviewer checked and confirmed that the consumer the agent found was live and active. So this was not a theoretical risk. The PR waited

修复的关键在于顺序，而不是拒绝变更。下游表首先被更新，使其列能够容纳更宽的类型，只有在那之后，我们的 PR 才进行合并和部署。没有发生故障。第一层中的脚本显示安全。Agent 发现了漏洞，而人工做出了决断。

英文原文

the fisk the fix was about order not about rejecting the change. The downstream table was updated first so its column could handle the wider type and only then did our PR merge and deploy. No incident. The scripts in layer 1 said safe. The agent found the gap and a human made the call.

这并不是一次性的事情。大多数变更都可以由 Agent 放行。当检查通过且 Agent 对其结论确信时，它会自行批准。但有一整类变更，Agent 永远不能给予批准。无论它听起来多么自信，

英文原文

And that was not a one-time thing. Most changes can be cleared by the agent. When the checks pass and the agent is confident in its verdict, it approves on its own. But there's a whole category of changes where the agent never gets to say approved. And no matter how confident it sounds,

涉及敏感数据的授权就是其中之一。谁能访问敏感数据，是我们希望每次都由人工把关的决定。

英文原文

grants touching sensitive data are one. Who can reach sensitive data is a decision we want a person to own every time.

看起来改变了意图而不仅仅是机制的迁移，是另一种情况。如果一个变更改变了数据的含义，而不仅仅是其移动或存储方式，由人来决定。

英文原文

A migration that looks like it changes intent, not just mechanics, is another. If a change alters what the data means rather than just how it is moved or stored, a person decides.

如果 Agent 本身对自己的结论没有把握，那是第三种情况。它不能仅仅因为没有查出具体错误就四舍五入直接批准。在这全部三种情况下，它的工作仅仅是指出风险及其推理依据。决定权依然在人工手中。

英文原文

And if the agent itself is not confident in its own verdict, that's the third. It does not get to round up to approve just because nothing concrete came back wrong. In all the all the three cases, his job is only ever to surface the risk and his reasoning. The decision stays with a person.

这是一个 PR。现在，这如何在每个 PR 中保持一致，审阅者又是如何随着时间的推移变得越来越好的？

英文原文

That is one PR. Now, how does this hold up across every PR and how does the reviewer get better over time?

我们如何知道标记是否有用，或者规则变更是否有帮助？我们从审查评论中 Agent 的结论及其推理依据开始。

英文原文

How do we know the flags are useful or whether a rule change helps? We start with the agents verdict and its reasoning in the review comments.

人工回复表示同意或不同意，并解释原因。正是这种解释使得反馈具有实用价值，而不仅仅是一次投票。

英文原文

A person replies agreeing or disagreeing and explain why. That explanation is what makes the feedback useful rather than just a vote.

这些回复每天都会被收集。值班人员每周阅读一次反馈，并手动更新审查规则。这些规则指导下一次审查。

英文原文

Those replies are collected every day. Once a week, the on call reads the feedback and updates the review rules by hand. Those rules shape the next review.

我知道这目前是人工的。没有任何东西会自动重新训练，也没有规则会自动更改。但我们有一个想法，让事故调查 Agent 协助进行每周的检查。但它还没有构建出来。

英文原文

I know this is manual today. Nothing retrains itself and no rule changes automatically. But we have an idea to let an incident investigation agent help with that weekly pass. But it is not built.

如果没有反馈，每一次规则变更都将是一次猜测。那么让我向你们展示一个例子，其中

英文原文

Without the feedback, every rule change would be a guess. So let me show you an example where a

人工纠正了智能体，而我们利用该反馈来改进它的下一次审查。

英文原文

human corrected the agent and we use that feedback to improve its next review.

这里有一个真实的例子。注意角落里的标签。这是一个单独的例子，不是我们的模式变更 PR。

英文原文

So here is a real one. And note the label in the corner. This is a separate example, not our schema change PR.

在源数据库中，某一列被标记为个人信息。但当它存入数据湖时，摄取管道已经对其进行了加密。因此该 PR 提议在数据湖中将其标记为非敏感，尽管上游数据仍然是 PII。

英文原文

A column was marked as personal information in the source database. But by the time it lands in the data lake, the injection pipeline has already encrypted it. So the PR proposed labeling it nonsensitive in the data lake even though the upstream data is still PII.

智能体对此提出了警告。从它的角度看，上游标记为 BII，所以非敏感标签看起来是不对的。作者提出异议，并要求智能体去查看其他表上的类似列之前是如何分类的，而不是单纯依据上游的标签。

英文原文

The agent flagged this. From where it stood, the upstream says BII, so a nonsensitive label looked wrong. The author pushed back and asked the agent to look at how similar columns on the other tables were already classified instead of going purely off the upstream tagging.

一位人工审查员同意了作者的看法。写入前的加密使得该列标记为非敏感看起来是可行的。这种纠正正是让规则变得更聪明的信号，因为下一次出现类似列时，这一上下文就能阻止一次不必要的呈报升级。

英文原文

A human agreed with the author. The encryption before write made the nonsensitive label look okay for this column. And that kind of correction is exactly the signal that teaches the rules to get smarter because the next time a similar column comes through that context can stop a needless escalation.

不过需要提个醒：这取决于具体管道在写入前进行了加密。并不是说“加密总是会让数据变成非敏感”这样一条一概而论的规则。

英文原文

One caution though, this depends on the specific pipeline encrypting before the right. It's not a blanket rule that encmpion always makes data nonsensitive.

那么，我们如何知道审查工具是否真的表现良好呢？

英文原文

So how do we know whether the reviewer is actually doing a good job?

我们关注两类错误：它批准的有问题的 PR，以及它不必要地发送给人工的没问题的 PR。这一矩阵划分出了四种可能性。

英文原文

We look at two kinds of mistakes. Bad PRs it approved and fine PRs it unnecessarily send to a person. That matrix separates four possibilities.

没问题的 PR 获批，以及有问题的 PR 被呈报，是两种正确的结果。

英文原文

A fine PR approved and a bad PR escalated are the two correct outcomes.

有问题的 PR 获批属于漏报。这很危险，因为变更上线了。

英文原文

A bad PR approved is a miss. That's dangerous because the change ships.

没问题的 PR 被呈报则属于误呈报，这会耗费人工时间。一个将所有内容都呈报的门禁可能会避免漏报，但并没有减少审查负荷。

英文原文

A fine PR escalated is a false escalation that costs a person's time. A gate that escalates everything may avoid misses, but it has not reduced the review load.

截至 2026 年 8 月 31 日，我们的漏报率大约是 2%。

英文原文

As of August 31st, 2026, our miss rate was about 2%.

大部分源于规则中的遗漏，而不是判断力的失误。我们没有单一明确的误呈报数据。但最主要的原因是缺少上下文，就像我刚才给你们举的加密列例子那样。

英文原文

Mostly from the gaps in the rules rather than the failures of the judgment. We don't have a clean single false escalation number. But the dominant cause is missing context like the encrypted column example that I just gave you.

漏报让我们获得新规则，而误呈报向我们展示审查工具所需的上下文。这里就是其中一个漏报案例。

英文原文

Misses give us new rules and false escalation show us context the reviewer needs. So here is one of those misses.

再次请注意标签。这是一个单独的例子，不是我们的模式变更 PR。

英文原文

Again know the label. This is a separate example, not our schema change PR.

我们有一份某部分数据的公开副本。其中的某些列即将被重新打标，并且有一个下游读取端从该公开副本中读取这些列。

英文原文

We have an open copy of the sum of some data. some of its columns are about to be retagged and there is a downstream reader that reads those columns from the open copy.

隐私团队将这些列重新标记为个人信息。一旦发生这种情况，它们就会移入数据的一份更受限制的副本中，并从公开副本中消失。

英文原文

Privacy retag those columns as personal information. When that happens, they move into a more restricted copy of the data and they are gone from the open copy.

相关的 PR 看起来没问题，而且合并了。但没有人检查的是，下游是否有任何服务仍从原先的公开副本中读取这些列。

英文原文

The red PR looked fine and it merged. But what nobody checked was whether anything downstream was still reading those columns from the old open copy.

确实有服务在读。一旦这些列消失，那个读取端就崩溃了。这就是一次漏报。

英文原文

something was. And once the columns were gone, that reader broke. That's a miss.

一个本该呈报的 PR 被批准了。

英文原文

A PR that should have been escalated got approved.

这不是判断力的失误，而是规则中的漏洞。这是一个先前无人写下的边缘情况。于是，它变成了一条新规则：当一列被重新标记为敏感信息时，在代码库中搜索是否有任何人仍在从即将失去该列的副本中读取它。规则就是这样变得更完善的。漏报被发现了。

英文原文

It wasn't a failure of judgment. It was a gap in the rules. A corner case nobody had written down. So, it became a new rule. When a column gets retagged as sensitive, search the code base for anyone still reading it from the copy that's about to lose it. And that's how the rules get better. The misses get found.

Heat.

Heat.

Heat.

Heat.

英文原文

Heat.

Heat.

Heat.

Heat.

Jared 介绍 Bun 与 AOT 编译器

大家好。我叫 Jared。我是 Bun 的创建者。我在 Anthropic 工作。Bun 是为 Claude Code 提供支持的 JavaScript 运行时。最近我合并了一个包含数百万行代码的 PR，为 Bun 添加了一个预先编译器（AOT compiler）。基本上，其核心思想是不在运行时承担 JIT 的开销，而是可以提前把所有代码都编译好。

英文原文

Hello everyone. My name is Jared. Uh I'm the creator of Bun. Uh and I work at Enthropic. Um uh Bun is the JavaScript runtime that powers cloud code. And recently I merged a several million line of code PR um to add an ahead of time compiler uh to to bond which is uh basically the idea is uh instead of paying the cost of a JIT at runtime you can uh uh you can compile all the code ahead of time. Um

这是一个非常庞大的项目。在夏天的时候，我用 Rust 重写了 Bun。而这是一个规模更大的项目。这个预先编译器比那个项目规模还要大。

英文原文

and it's it's like a really big project. Um over the summer I uh rewrote bun and rust. Um uh and this is a project on a larger scale. This ahead of time compiler is like a larger project than that. Um and

我认为目前大型项目面临的挑战之一是：你究竟如何才能放心地发布由智能体编写的数百万行代码？

英文原文

I think one of the challenges right now with large projects is like how do you actually confidently ship a mill like millions of lines of code written by agents. Um

这就是我打算要讲的内容。

英文原文

so that's what I'm going to talk about.

在进行了一些讨论之后，最初的提示词大致是：我们基本上需要让 Claude Code 减少大量内存占用并变得更快，我们可以修改 WebKit 或 Bun。Bun 底层使用的是 JavaScriptCore，也就是 WebKit。到现在已经过去一个多月了。

英文原文

This was like the opening prompt after some discussion with it was like basically uh we need to make cloud code use a lot less memory and uh and faster and we can modify either webkit or bun. Um bun uses JavaScript core which is webkit um under the hood uh and it's been a little bit over a month. Um uh

这个编译器已经历了五次重写。你可以看到前几次重写都相当简短，然后我差点就把蓝色那个版本发布了。接着我觉得这构建时间实在太长了，编译代码花的时间太长了。所以又重写了一次。接下来的这一个版本将在下周发布。

英文原文

uh this is this compiler has gone through five rewrites. Um you can see the like the first couple ones were like pretty short and then I like almost shipped the blue one. Uh and then I was like this is building way too long. Like it's taking way too long to compile the code. So rewrote it again. Uh and this next one is shipping next week. Uh

我对类似这样项目的思考方式大致是：你真正受制于你如何测试代码能正常工作。所以这甚至比你具体……比如你选用哪种方式去构建它更重要，因为这完全取决于你如何验证它确实能正常工作、没有 bug、并且你是可以信赖它的。

英文原文

this is kind of the way I think about a project like this is like you're really constrained by how you test that the code works. Um so that's like even more important than what you're specifically like like the approach you go with for how to build it is entirely driven by how do you verify that it actually is going to like work and not be buggy and that you can trust it.

再补充一点关于为什么做这个特定项目、为什么做预先编译器的背景：这是因为对于典型的基于 JavaScript 的应用程序，大致有一半的 CPU 时间都只是花在了按需编译代码上。

英文原文

Um and so just a little more background on like like why this particular project why the ahead of time compiler and it's because uh for a typical JavaScript based application roughly half of like this the CPU time is just spent compiling code on demand.

因此，大家在许多应用程序中见到的普遍现象是，人们用像 Rust 或有时用 Go 这样的语言来重写他们的代码库。从 JavaScript 转过去的时候，原因在于这确实是 JIT 加上单线程应用程序特性的共同影响。

英文原文

Um and so a lot of the general like thing you you've seen in a lot of applications where people are rewriting their code bases in in a language like Rust or sometimes Go um it's like when coming from JavaScript it's because uh like there it's really the the combination of the JIT and that it's a single threaded application.

这就是 JIT 和 AOT 之间的大致区别：AOT 是提前编译，而 JIT 是边运行边编译。

英文原文

Um so that's sort of the difference between a JIT and AOT is that uh you compile before whereas with a JIT you compile as you go along.

这对于像 Claude Code 这样的项目尤为重要，比如使用 Claude Code 的 iOS 应用时，CPU 时间非常……如果你让 Claude Code 的每一次会话也都在进行编译的话。

英文原文

This is particularly important for something like cloud code where when using like the cloud code like iOS app where the CPU time is a lot uh if you're if you have to have every session of cloud code also compiling

所以我们深入讨论过很多次：我们是否应该用 Rust 重写 Claude Code？因为 Bun 团队——也就是我们——用 Rust 重写了 Bun，所以我们在重写方面相当在行。而我们决定不这样做。

英文原文

um so we talked a lot about like like should we rewrite cloud code in rust um because the bun team us rewrote bun in rust So like we're pretty good at rewrites. Um uh and we decided not to.

原因在于，对于 Bun 而言，进行 Rust 重写其实相对容易，Bun 是一个容易进行 Rust 重写的项目，因为测试是用一种独立于其运行环境的语言编写的。由于所有测试都是用 TypeScript 编写的，我们可以在 Zig 版本和 Rust 版本上都运行这些测试；而对于 Claude Code 来说，Claude Code 的测试是用 TypeScript 编写的。因此我们必须移植 Claude Code 的测试套件，而一旦你开始重写测试，你就无法真正再信任这些测试了。所以那看起来风险太高了。

英文原文

Um and the the reason why is because like for in bun's case it's actually like pretty easy to do like bun is an easy project to do a rust rewrite because the tests were written in a language that's independent of the one it runs in. But so because all the tests were written in Typescript um we could run the tests on both the zig version and the rust version whereas for cloud code uh we like cloud codes tests are written in typescript. So we would have to port cloud codes test suite and as soon as you're like rewriting the tests then you can't really trust the tests anymore. Um so that seemed like it'd be too risky.

所以退一步讲，要是我们直接让 JavaScript 变得更快，从而不必用 Rust 重写它呢？

英文原文

So instead, what if we just make JavaScript faster so we don't have to rewrite it in Rust?

这是稍后一点的另一个提示词。

英文原文

Um, this was another prompt a little bit later. Um, uh,

结果是它确实把 JavaScript 的内存——比如内存占用——减少到了大约四分之一。它的启动速度大约快了 2 倍。我们正在内部进行内部试用（dogfooding），很快就会对外开放。

英文原文

and the result is that it actually did uh have JavaScript's memory like the memory usage uh, it uses about 4x less memory. Um uh it starts about two times faster. Um uh and we we're dog fooding this internally and it'll be external soon.

还有一些基准测试，稍后我也将更详细地介绍我们究竟是如何做到这一点的。一件有趣的事情是，TypeScript 7 手工将他们的代码从 JavaScript/TypeScript 重写成了 Go，而通过 bun compile 进行编译的单线程版本，现在比单线程的 TypeScript Go 移植版还要快。我们也会为 Bun 的预先编译版本添加线程支持，但我们目前还没做。因此已经有一个 PR 了。

英文原文

Um some more benchmarks uh and then I'll also go into more detail about how we actually did this. Um so like it's it's possible that if an interesting thing is like like TypeScript 7 rewrote their code by by hand um from uh JavaScript/typescript to to go and uh the singlethreaded version uh with you when compiled via bun compile is now faster than the singlethreaded TypeScript go port. Um, and we will add threads also to the ahead of time compiled version of bun, but we haven't done that yet. So there is a PR.

关于这个实际的预先编译器的一点细节：TypeScript 并不是类型……类型并不一定需要是正确的。因此它把类型当作预测，而不是直接信任它们。

英文原文

Um, and so a little bit about like the actual ahead of time compiler. Um, uh, TypeScript is not type types are don't necessarily need to be correct. So it treats types as predictions, uh, instead of like trusting them.

这类问题非常适合模型来处理，因为它们就是非常擅长编写编译器。

英文原文

Um, and this kind of problem works really well for models because uh they're just really good at writing compilers. Um, so so uh

这更关乎我们当时是如何思考这项决定的。这里的一个重要因素是，当它处于预先……时，你可以运行非常多的测试

英文原文

uh this is sort of like more about like how how uh like should we like how we were thinking about this decision. Um, and a a big thing here is like there's just so many tests you can run when you're when it's like uh in ahead of

因为像这种事情，我们自己几乎不需要写任何测试。我们实际上确实写了很多测试，但大体上我们可以相信在项目之前就已存在的那些测试。在用 Rust 重写 Bun 时也是一样，当时我们有 Bun 现有的测试套件可以运行；而现在有了编译器，我们有 Bun 现有的测试套件，加上 WebKit 的测试，加上 Test262，再加上 Claude Code 的测试。

英文原文

time like like for something like this because we don't need to write any tests pretty much ourselves. Like we actually do write a lot of tests but uh we can largely trust that the tests that predate the project work. Um, and this was the same thing with the the bun rust rewrite where we had buns existing test suite we could run and now with a compiler uh we have bun's existing test suite plus web kids test plus test 262 plus cloud codes tests

然后我们还做到了让 Bun 本身、也就是 Bun 自己的测试运行器能够在 AOT 模式下运行。

英文原文

um and then we also made it so that way bun itself the bun's own test runner could be run in the AOT mode uh

而且在提前编译器（AOT compiler）上我们有一个破坏性变更，或者实际上是两个破坏性变更，这基本上就是为什么这些特定测试会失败的原因。不过这两个破坏性变更实际上主要是一些安全方面的东西，用来防止原型污染攻击（prototype pollution attacks），但它们最终也对性能有好处。

英文原文

and like the we have one breaking change with or really it's two breaking changes with the ahead of time compiler and that's pretty much why these particular tests are failing. Um but the and the two breaking changes are actually like mostly security things um to prevent prototype pollution attacks but they also end up benefiting for performance. Um

所以，让这种特定类型的问题非常适合智能体（agents）的原因在于，你可以同时运行一个应用程序的全部三个版本：一个带 JIT，一个带提前编译，还有一个不带类型。只要这三个版本的行为一致，那你至少就知道它在这个特定应用程序上是可以正常工作的。但随后你可以把这种问题拿去进行大量模糊测试（fuzz），让 Claude 大量生成代码生成器去对不同模式进行模糊测试，通过这种方式你可以捕获大量的 bug。

英文原文

so like what makes this particular kind of problem pretty good for agents is like uh you run it with you can run all three versions of an application. One with a JIT, one with ahead of time compiled and one also without the types and as long as the three of them uh behave the same then you know that it works at least for that specific application. But then you can take that kind of problem and fuzz it a lot. Um uh and just have Claude like turn out code generators to like fuzz different patterns. Um and you can catch a lot of bugs that way. Um

编译器重写历程与智能体工程组织

这是一个非常庞大的项目。从代码量来看，我认为这是我做过的最大的项目。其中大部分是测试，而且也因为我们把它重写了五次。

英文原文

uh this is a very large project. This is the biggest I think I've worked on. Uh in terms of like code volume, um most of this is tests. Um and also because we rewrote it five times. Um

在做这种项目的时候，我的思考方式是：你会有一批批的阶段，比如“好，我们现在需要一次性写很多代码”；而在那最初的一批之后，就会变成“好，让我们修复所有问题，确保能编译通过，确保通过大量的测试，并确保它能真正切实地工作”。具体的组织运作方式是，当你处于大批量编写的项目阶段时，你不想频繁运行构建，你只想运行一次构建，因为否则你就会把所有时间都花在编译和等待构建错误上，这不会是让任何人有效利用时间的方式。另外，在更长期的运行中，一旦你把所有代码都写完了，你就会希望拥有许多独立的智能体。

英文原文

when you're like doing this kind of project, what how I think about it is like you have like batches of like okay we need to write a lot of code at once and then after that initial batch then it's okay let's have let's like fix everything make sure it compiles make sure it passes a bunch of tests um and make sure it like really truly works. Um, and the actual logistics of that are like when you're in the big batch kind of project stage, then uh you don't want to run builds, you want to run one build because otherwise you're going to spend all your time compiling and like waiting for build errors and it's going to like not be a good use of anyone's time. Uh and then separately uh for like longer running like once you have once you've written all the code uh you want to uh have it like have lots of individual agents. Um

这是这个项目的一些时间线。第一个版本不是很好。Claude Code 的体积达到了一个千兆字节（gigabyte），这太糟糕了。而且这个版本尽可能多地复用了 JavaScriptCore 的现有代码，所以它的上限基本上受限于能从那里得到的程度。于是在这之后不久，我们在 Bun 本身中加入了一个 TypeScript 类型检查器。

英文原文

this is like a little bit of the timeline um of this project. Uh the first one was not very good. uh like cloud code was a gigabyte um which is terrible uh uh and like this used as much as possible from JavaScript core's existing code um so it was like basically the ceiling was what is is like only as good as like what it could be from that um uh so then a little bit after this we added a TypeScript type checker into bun itself

这些是提示词。我们加入了 bun check，然后就对 Claude 说：“嘿，Claude，请使用 bun check。”在那之后，启动时间加快了 2 倍，在 Claude Code 中开始少用大约 3 倍的内存，但是生成的代码是烂代码（slop）。

英文原文

Um these were prompts uh uh and so we added bun check and then we're like hey claude please use bun check. Um uh and then after that startup time got two times faster and uh in cloud code and uh started using about 3x less memory but the code was slop

这可不好。它绝不应该——基本上这块的意思是，任何访问名为 archive 属性的对象，它都会推断可能使用了 bun.archive API。我看到这个之后就觉得，我们绝不能发布这个东西，这不行。

英文原文

uh and like like this is not good. it should never basically this meant that any object that accessed a property called archive it would then assume that the bun.archive archive API was potentially used. Um, and I looked at this and I was like, we cannot ship this. This is not good. Um

所以然后，还有另一件事就是，当你在做大型规划、开始让它写一堆设计文档之类的东西时，它就是极度热衷于写 Markdown 文档，但这根本讲不通，因为谁会想看——1500 个 Markdown 文件根本就是不合常理的。

英文原文

so then uh uh this is like yeah and this is another thing too is like it just like to like when you're doing like a large planning like when you're starting to like have it do a bunch of design docs and stuff it just loves writing markdown docs and it doesn't make sense because who's going to like 1500 markdown files else is just not reasonable. Uh uh

所以针对这个，我想这是第三个编译器版本，也可能是第四个，不，这是第三个。为了像大批量处理一样把代码全弄出来，140 个智能体同时把所有代码一口气全写完，然后让 Claude Code 跑起来。

英文原文

so for this it was like and this was like I think the third compiler version maybe it was the fourth. No this was the third. Um uh it to like just to kind of batch get it all out uh 140 agents at the same time write all the code all at the same time. then get cloud code to run. Um

然后，这是第四个版本。这一个版本的代码不再是烂代码了，相当不错，性能基本上和以前一样。然而编译时间实在太慢了，因为来自 JavaScript 背景的人完全不期望有任何编译时间。对于 Claude Code 这样一个相当庞大的代码库来说需要 50 秒，但这意味着其他项目需要几秒钟（个位数秒），而人们不想等那么久。

英文原文

uh and then uh this is the fourth one. Um and this one the code was not slop um pretty good um with like basically the same performance as before. However, the compilation time was way too slow because people coming from like a JavaScript background don't expect any compilation time. Um uh so 50 seconds for like cloud code is like a pretty large codebase but this meant that like other projects would take like single-digit seconds and people don't want to wait that long.

所以我们要再次重写它。因为在编译器的较早版本中，它只需要大概 10 到 20 秒。

英文原文

Um so uh we're going to rewrite it again. Um and and uh because in in older versions of this compiler, it was taking like 10 to 20 seconds.

然后 Claude 就会做这么一件事：它会试图去评估一切，以判断是否值得去做，但是当你的度量并不好的时候，你很容易自我欺骗说自己的度量是好的。其中一部分原因还在于，为什么这会有 90 万行代码？这应该小得多才对。

英文原文

Uh and then cloud does this thing where like it'll try to like size up everything to judge whether or not it's worth working on, but it's really easy to convince yourself that like your measurements are good um when they're not good. Uh uh and like part of it was also like why is this 900,000 lines of code? This should be a lot like smaller. Um

于是我们又做了一次。这次代码很好，构建时间在 20 秒以内。

英文原文

uh so then we did it one more time. Um and the code was good and the build time uh was under 20 seconds.

未来某个时候我们可能会搞出第六次重写，但第五个版本已经非常适合发布了。而且我已经在这个上面投入了足够长的时间，是时候把它发布出去了。

英文原文

Um, it's possible that we end up with a sixth time sometime later, but the fifth one is like very shippable. And I've worked on this for enough time that it's like time to ship it.

这里面很多过程就像是直接说：“嘿，Claude，试试这个想法，然后看看会发生什么。”然后它就会去评估那个想法。我觉得这仍然有点不太对劲。我认为其中很多都是我们应该再去尝试的事情，不要让它——有点像我之前说过的，评估一个改动的影响时，很容易低估或者评估错误，然后把自己给劝退了。所以我觉得这可能需要重新审视一番。

英文原文

Um, a lot of this was like just like, "Hey Claude, try this idea and then see what happens." And then it would like size up the idea. Um, and I think this is like still kind of not right. Uh, like I think a lot of this are like things that we should try again and not let it like kind of what I said earlier about like uh sizing up the the impact of a change is like it's really easy to underest or to like size it up incorrectly and then talk yourself out of it. Um, so I think this probably needs to be revisited more.

我以前的方法，特别是在用 Rust 重写 Bun 时，非常依赖 Claude Code 的工作流来做事。而现在换成最新的模型后，我基本上主要依靠长生命周期的子智能体（subagents）。除了最开始那个一次性把所有代码都写出来的那一部分。对于子智能体来说，这样运作得相当好，因为它们能把相关代码的所有上下文都记在脑子里，或者说保留在上下文窗口里。那样浪费更少，而且它们可以一直推进下去。

英文原文

Um my old approach particularly with like the the bun rust rewrite was very much leaning on like cloud code workflows to do things. Um and pretty much now with the most recent models uh I mostly just rely on sub agents that are like very longived. Um except for that initial part where you're like writing all the code all at once. Um, and this with the sub agents, it works pretty well because they keep all the context of the relevant code in their head or like in the context window. Um, uh, and uh, that's like less wasteful. Um, and like they can just keep going. Um,

这里有关于编译器不同部分如何工作的更多细节。

英文原文

here's a little more about like how the different parts of the compiler works. Um

通常是这样的：如果我们能确定证明某个值——比如类型是相同的、它们是单态的（monomorphic），那么它就可以变成字段访问；否则它每次都必须去检查。它有一些内联缓存（inline caching），但不是很多。

英文原文

generally it's like there's if we can confidently prove that uh the value is like the types are the same that they're monomorphic then it can become a field access otherwise it has to check every single time uh uh and it has like some inline caching but not like a lot. Um

面对这种规模的项目，有时候确实很难跟进所有正在发生的事情。所以我非常依赖每小时更新一次。它会发布一份记分卡，对比我们优先关注的一系列指标。

英文原文

uh as with a project of this size, you really do need to it's kind of hard to follow sometimes like what everything what everything's happening. So uh I leaned a lot on like just having an hourly update. Um and where it would post a scorecard comparing a bunch of metrics that we were prioritizing.

不过通常在 Slack 中发布时，它会展示上一个版本和当前版本，这样就很容易看出数字是上升了还是下降了。但在幻灯片里这样展示没什么用，因为前一个版本就只是原本的 JIT。

英文原文

Um, usually though when it was posting in Slack, it would show like a previous and this version so that way it was easy to see if numbers went up or down. Um, but it's like not helpful in a slide because the previous one is just legit.

重要的一点是，这些测试在很大程度上没有一个是编造出来的。它们是早于项目就存在的东西，这样我们就知道智能体不可能篡改过它们。Fuzzilli 是 V8 使用的一个 JavaScript 引擎模糊测试工具。我们在 Bun 中已经在使用它了，但我们也对它进行了扩展，让它同样能用于测试 TypeScript 类型。

英文原文

Um, the the important thing is like none of these tests for the most part are things that are like made up. Um uh they're things that like pre predated the project. So that way we know that like the agent couldn't have messed with it. Fuzili is like a JavaScript engine fuzzer that V8 uses. Um so uh and we were already using that in bun but we also extended it to have uh to like test for TypeScript types um as well. Um

而且检查测试是否真正能捕获问题也很重要，因为当测试没有发现 bug 时，你很容易自欺欺人地以为它能找到 bug。

英文原文

uh and it's also important to check that the tests actually catch things because it's really easy to fool yourself that it's finding bugs when it's not finding bugs.

提前编译器的一些更多数据。在大多数项目上，它通常表现优于 JIT。不总是如此。大家可以在 uglify.js 的案例中看到，如果它必须……

英文原文

Some more numbers uh on the ahead of time compiler. Um it generally for most projects outperforms the JIT. Um not always. the you can see in the uglify.js case and that's basically if it has to

大量使用解释器。因此，如果有大量类似 eval 或 new Function 的操作，那么它就不会更快。不过大多数应用程序并不会这么做。这就像……这就像是尽可能接近一行代码就能让应用程序运行得更快的做法。这并不是 100% 成立的，因为如你所见，有 UglifyJS 以及其他情况存在。但通常情况下确实是成立的。

英文原文

use the interpreter a lot. So if there's a lot of like eval or new function um then it's not faster. Um but most applications don't do that. Uh and it's like it's like a make it like it's as close as possible as you can get to like a oneliner to like make an application run faster. Um this is not 100% true because as you can see uglify js and there are other cases as well. Um but it is often true.

另外一点就是 Object.prototype。这属于罕见的极端情况，但基本上 JavaScript 应用程序中存在很多被称为原型污染（prototype pollution）的安全漏洞。这通常是通过向 Object.prototype 赋值属性来实现的。而编译器禁止了这种做法。

英文原文

So like yeah the other thing is object prototype. Uh this is like a rare edge case but basically there's a lot of security vulnerabilities in JavaScript applications called like prototype pollution. And this usually works by assigning properties onto object.prototype. Um, and the compiler bans that.

那么对于这样一个大项目，你整天实际都在做些什么呢？这些是我发送的最多的消息，频繁地说“继续”。另外，我的笔记本电脑老是没电。

英文原文

So like for a large project like this, what do you actually do all day? These were like the top messages that I sent saying continue a lot. Um, uh, also like my laptop kept dying. Um uh

不过更具体地说，你实际做的是提出一堆引导性问题，比如“我们为什么要这么做？”、“我们能这样做吗？”此外我经常会说“你能快点做这个吗？”或者“为什么花这么长时间？”，因为它总有点像是在自说自话地编造规则。而且它往往倾向于低估自己能做的事情。所以我经常对 Claude 说这类话，促使它去更加努力地尝试。

英文原文

and then like but like more concretely what you actually do is like a bunch of leading questions like why are we doing this? Uh can we do this? Also I end up saying like can you do this faster or like what's taking so long because it kind of like talks itself into making up rules. Um uh and also it tends to not be uh it tends to like underestimate what it can do. Um so I often end up saying things like this to Claude to to get it to like uh try harder.

结果就是在 JavaScript 引擎主要使用的基准测试之一 JetStream 3 中，Bun 在其中绝大多数测试项上的表现都超越了 JavaScriptCore 的 JIT 以及 Node.js（即 V8 JIT）。而且整个过程中，一直都在使用 Claude Code 来构建 bun compile 本身，然后使用通过 bun compile 编译构建出的 Claude Code。

英文原文

And the result is that uh Jetream 3, which is one of the main benchmarks that JavaScript engines use, um uh the bun outperforms both JavaScript cores JIT and no Node.js in like V8 JIT um in most of them. Uh, and the whole time like it's been using cloud code to like build the bun compile like to build bun compile itself and then using the build of cloud code that was compiled with bun compile.

我对这件事情的看法大概是这样：在与 Claude 合作开展大型项目时，可测试性这件事非常重要，必须确保测试确实测到了东西。如果你不积极地重新测量，情况就会变差，因为涉及的内容太多，很容易跟丢。对于这类尚未发布的大型项目，重写往往比重构更容易。

英文原文

Um uh so sort of how I think about this is like like the testability thing is super important when you're working on a large project with claude. Uh and to make sure that the tests actually test things. Um and if you're not actively reme-measuring then it's going to like get worse because it's just if there's so much stuff going on it's really easy to lose track. Um uh and like it's easier for large projects like this that haven't shipped yet to uh rewrite than it is to refactor.

如果你不抱有雄心去行动，模型带来的改进就会开始变成 0.1%，而不是 10%。所以你必须不断督促它：“嘿，只要你更加努力去尝试，就能做出非常酷的事情。”如果你安装了 Bun，现在就可以尝试一下，运行 bun upgrade --canary，然后直接运行 bun compile 即可。谢谢大家。

英文原文

Um and if you don't act ambitiously then the models will like start having like 0.1% improvements instead of like 10% improvements. So you kind of have to constantly nudge it to like hey you can do really cool things if you try harder. Uh you can try this now if you want uh by running bun upgrade- canary if you have bun installed um and then just bun compile. Uh thanks everyone.

[掌声]

英文原文

[applause]

[掌声]

英文原文

>> [applause]

换麦。换麦。换麦。换麦。换麦。换麦。换麦。换麦。换麦。换麦。大家好，我们在这里为大家带来演讲——“AI 规模下的代码合并”。这实质上是 Uber 从保持主干完全处于绿色状态，转变为出现某种程度破损主干的历程。我是 Priam Devi，Uber 开发者平台团队的 Staff 工程师。

英文原文

>> Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Heat. Uh hello everyone. Uh we are here to present our talk uh merging at um AI scale. Uh this is essentially uh Uber's journey from going to a fully green trunk to somewhat broken trunk. Um I'm Priam Devi. I'm a staff engineer at Uber working in developer platform.

大家好，我是 Majeri，是与 Priam 一起在 Google 开发者……工作的 Senior 工程师。

英文原文

>> Hi, I'm Majeri. I'm a senior engineer working alongside Priam uh in the developer at Google.

好的。这是本次演讲的提纲。我们将探讨 Uber 目前的代码合并是如何运作的，接着介绍 AI 如何改变了这一格局，随后介绍我们的应对举措以及如何进行适应，然后讨论各项成果，并聊聊我们接下来的规划。

英文原文

>> Great. Um so here's the outline for the talk. Uh we're going to go over like how the merging um works at Uber at the moment and then how AI changed the landscape and then I'll respond um to that how we adapted and then discuss the results and um talk about what are we planning next.

Uber 的单体仓库与合并队列

在 Uber，我们有 6 个基于语言的 monorepo，超过 5,000 名开发者每周贡献超过 36,000 次 commit 和 27,000 个 PR，并且对服务和库的绝大部分更改都是在这些 monorepo 之一中完成的。在本次演讲中，我们在讨论任何内容时都会主要聚焦在 Go 代码上，因为那是我们最大的一个代码库，也是我们面临绝大多数扩展性挑战的地方。

英文原文

So at Uber uh we have a six language based mono repos where more than 5,000 uh developers uh are contributing over 36,000 commits uh 27,000 PRs uh every week and uh pretty much every changes uh to the services libraries are being done in one of these monor repos. uh for the purpose of this talk we're going to focus mostly on go code when we talk about anything because that is our the biggest one and that's where the most of the scaling challenges uh we face.

我们来理解一下这个问题，弄清楚为什么我们这里需要一个合并队列。这里基本上是有两位用户，用户 A 和用户 B，在主干的一个 base 边缘上分别创建了 PR A 和 PR B。当创建 PR 并运行 CI 时，CI 是绿色的，构建通过，一切正常。

英文原文

So let's understand the problem a little bit and um and understand like why we need a merge cube here. So uh given here basically we have two users uh user A and B creating PR A and B um on a base edge uh from the trunk and when the PRs are created and CI is run uh the CI is green bills are passing everything is good.

现在，当 A 被合并到主干中时，我们可以看到合并 commit 的构建仍然是绿色的，构建依然通过；但当 PR B 被合并到主干中时，构建不再是绿色的了。原因是 B 从未与 A 合并的更改一起进行过测试。我们在合并前从未测试过这两者的组合，因此当 B 合并时就坏掉了。

英文原文

Now uh when a uh is merged into the trunk uh we can see the bills for the merge commit are still green u bills are still passing but when the PRB is merged into the trunk um bills are no longer green and the reason is that B was never tested uh with the changes that were merged by A. So uh we never tested the the combination of those two things before merging and hence the the when B is merged it's broken.

现在设想一下，当有数千名工程师在同一个 monorepo 中工作，每个人每天都在进行数百次乃至数千次这样的更改。这就成了一个持续存在的问题：主干非常频繁地损坏，造成了修复的负担，并在全公司范围内带来了排查和修复的损耗。

英文原文

Um now imagine when we have thousands of uh engineers working in a monor repo and everyone is making changes u like hundreds and thousands of those every day. Uh this is a constant problem uh where trunk is broken um very frequently and there is a burden on fixing and there is a toll across the company to sort of um figure it out and uh fix it.

针对这个问题，我们在这里介绍一种解决方案，那就是 Summit 队列。这是我们部署用来解决此问题的 Uber 合并队列。在这种情况下，我们可以看看两个示例 PR，PR A 和 PR B。这里我们实质上在做的是预测在 main 分支上可能成立的未来状态。这里存在三种情况：A 单独合入 main 分支，B 单独合入 main 分支，以及 A 和 B 的组合。总的来说，我认为我们想要做的是合入能够让主干整体保持绿色的更改。在这个例子中，当我们把 A 和 B 一起合入时，主干是绿色的，这就是 Summit 队列的大致运作方式，从而让我们的 monorepo 保持主干绿色。

英文原文

Um so here uh we're going to present a solution uh to that problem uh and that solution is summit Q. Um that is our Uber's merge Q um that we deploy uh to solve this problem. And in this case we can see our two example CRP A and PRB. And here we essentially what we are doing is we are predicting the futures that would be uh that could be true on main. So there are like three things here. A could land on on its own to the main B could land on its own to the main and then the combination of A and B. In general I think uh what we want to do is that we want to land the change that is going to keep the trunk green as a whole. So in this case when we land A and B together the trunk is green and uh and that's how basically uh sort of summit Q works and keeps the trunk green for our monor repos.

那么在这一版本中它是如何扩展的呢？Summit 采用了三项机制。第一项是目标分析（target analysis）。这意味着当两位用户在两个不同的服务中工作时，他们的更改并没有任何重叠。因此这两部分更改可以相互独立地进行构建和测试，然后再进行合并。这使我们基本上能够获得更高的吞吐量，而无需对进入队列的每一个更改都进行串行化处理。

英文原文

Now how does it scale in this uh version of it? Um there are three things that summit employs. U the very first thing is target analysis. Uh what it means is when two users are working across two different services uh they don't really have any overlapping changes. Um so those two things can be built and tested independently of each other and then can be merged. So this allows us to basically get higher throughput and not serialize every change that is entering into the queue.

第二项是概率推测（probabilistic speculation）。它的含义是，我们利用历史构建信息以及更改的特征，来预测给定更改的成功分数。然后我们借此来决定为了让该更改合入，可能需要哪些构建。因此在这种情况下，相比于推测产生的所有其他构建，我们优先考虑那些最可能需要、且对该更改合并到主干中至关重要的构建。

英文原文

Uh second is probabilistic speculation. Um what it means is uh we are using historical um the build information and also the the change characteristics to predict uh what is the success uh score for a given change. And then we are using that to basically decide what are bills are likely needed in order to land that change. And so in that case we prioritize the build that are most likely needed and are important for that change to be uh merged to the trunk rather than like every other build that uh comes from the speculation.

第三部分是旁路大变更（bypassing the large changes）。它的意思是，如果队列中有一个正在耗费较长时间构建的庞大变更链，而与此同时有较小的更改进入队列，这些较小的更改可以更快构建完成，因此它们无需等待大变更合入之后才能合入。这样一来，我们可以在不预先合入较大更改的情况下合入较小的更改，这项优化使我们基本上能够快速推进并获得更高的队列吞吐量。

英文原文

The third part is bypassing the large changes. Uh what it means is that if we have a large chain sitting in the queue which is taking longer to build and at the same time if we have a smaller changes entering into the queue um those can be built faster and so though those changes don't need to wait for large changes to land before they can land. So in this case we can land the smaller changes without landing the larger changes first and this optimization allows us to basically move fast and get a higher throughput of the queue.

在只有人类合入更改、创建 PR 时，这套机制一直运作良好。但随后我们迎来了 AI，PR 创建和合入的速度全然改变了。为了更好地理解这一点，我们来看一些数据。在 2026 年 1 月，我们大约每小时处理 90 个请求，而到了 3 月，我们每小时处理 130 个请求。也就是说在两个月内大约增长了 44%，而这一增长主要归因于 AI 生成的更改。

英文原文

So this is what has worked um when it was just humans uh landing changes creating PRs but um then we had AI and the changes at which the uh PRs were created and landed it all changed. So to understand it a better let's look at some of the numbers. So uh in January 2026 we were doing uh roughly 90 requests uh every hour and by March we were doing 130 requests every hour. So that's roughly a 44% increase in two months and the reason for this increase is primarily because of the AI generated changes.

鉴于系统的构建方式，这些更改的速率对其产生了影响。来看看这些情况：当我们每小时处理 90 个请求时，P95 排队时间为 30 分钟；而当我们每小时处理 130 个请求时，排队时间大约增加到了 3 到 4 个小时。在这种情况下，我们虽然始终在保持主干处于绿色状态，但是……

英文原文

Now given the way the system was built um uh it had the rate of uh these changes had effect on it. So um looking into those like we had 30 minutes of uh P95 line times when we were doing 90 requests every hour and that went to roughly 3 to four hours when we were doing uh 130 uh requests every hour. So in this case we were still keeping the trunk green uh at all times but

代价是开发人员基本上要等上三到四个小时才能把改动合入主干（trunk）。现在我把麦克风交接给 Murray，她会讲讲我们的应对措施，以及我们是如何适应这一局面的。

英文原文

at

the

cost

of

developers

essentially

waiting

for

three

to

four

hours

to

land

their

changes

to

the

trunk.

Um

now

I'm

going

to

hand

over

to

Murray

uh

and

she's

going

to

talk

about

our

response

to

it

and

how

we

adapted

uh

to

the

situation.

太好了。谢谢 Priam。刚才 Pritham 向我们介绍了几种 Submitu 现有的优化措施，这些措施到目前为止在应对人类产生的需求方面是有效的。但一旦 AI 加入进来，我们现有的优化措施就不再够用了。因此，我们的应对措施基本上是对现有的提交队列（submit queue）进行另一套优化，我们从三个不同的方向切入这一应对措施。第一种是我们引入了准入关控（admission gating）的概念，它帮助我们整形进入队列的流量。

英文原文

>> Great.

Thanks

Priam.

So,

Pritham

just

uh

walked

us

through

a

few

optimizations

that

Submitu

had

that

worked

so

far

in

keeping

up

with

the

demand

uh

that

humans

generated.

But

once

AI

came

into

the

picture,

uh

the

optimizations

we

had

were

no

longer

sufficient.

So

we

our

response

essentially

was

another

set

of

optimizations

to

our

existing

submit

queue

and

we

approached

the

response

in

three

different

ways.

The

first

was

we

introduced

this

concept

of

admission

gating

which

helped

us

to

shape

the

traffic

that

we

accept

into

the

queue.

接下来，我们引入了推迟（deferrals）的概念，这是我们引入的一个辅助队列，用来缓解由于准入关控给开发人员体验带来的痛点。最后，我们实施了所谓的冲突放宽（conflict relaxation），这有助于我们在队列内部重新换取更高的并行度。

英文原文

Uh

next

we

introduced

the

concept

of

deferrals

which

is

a

secondary

cue

that

we

introduced

that

helps

with

developer

experience

uh

from

the

pain

that

comes

with

admission

gating.

And

lastly,

we

implemented

what

we

call

conflict

relaxation

which

helps

us

buy

back

more

parallelism

uh

within

the

queue

itself.

接下来我们将稍作深入地逐一看下这些干预措施，确切了解它们每一项都在做什么。首先第一个，准入关控，基本上就如字面意思那样。在每个请求进入主提交队列的入口处设有一道关卡。我们会查看进来的请求，将其归入两类之一：要么是一个小型改动（small change），要么是一个高开销改动（expensive change）。至于其定义，可以是任何规则。

英文原文

Uh

next

we

will

look

at

each

of

these

uh

interventions

in

a

little

bit

of

depth

and

understand

exactly

what

each

of

them

are

doing.

So

the

first

one

admission

gating

uh

is

basically

what

it

sounds

like.

There

is

a

gate

at

the

admission

of

every

request

into

the

main

submit

que.

We

look

at

the

request

that

comes

in

and

we

categorize

it

as

one

of

two

things.

Either

it's

a

small

change

or

it's

an

expensive

change.

And

the

definition

of

this

uh

could

be

anything.

我们使用的一种定义是 Pam 早先在幻灯片中提到的目标分析（target analysis）。小型改动触及较少的目标（targets）。高开销改动可能是横跨多个目标、整个构建图之类的改动。所以更小的改动更有可能为你带来更高的并行度，我们基本上是让它在高峰时段准入，把它加入活跃的提交队列中照常构建和测试，然后继续往下走。但是一个高开销改动有可能因为占用资源、需要与队列中的其他所有改动一起验证，从而基本上变成一个瓶颈。

英文原文

One

of

the

definitions

that

we

use

is

the

target

analysis

that

Pam

talked

about

uh

earlier

in

the

slides.

Uh

a

small

change

touches

fewer

targets.

An

expensive

change

might

be

something

that

spans

across

uh

multiple

targets,

the

entire

build

graph,

something

like

that.

So

a

smaller

change

is

more

likely

to

give

you

higher

parallelism

and

we

basically

say

admit

it

during

peak

hours

and

add

it

to

the

active

uh

submit

Q

build

and

test

as

usual

uh

and

move

along.

But

an

expensive

change

has

the

capability

to

basically

be

a

bottleneck

by

holding

resources

uh

needing

to

validate

against

everything

else

that's

in

the

queue.

So

所以为了保护队列中的其余部分，我们在这个阶段所做的就是在高峰时段直接拒绝任何高开销改动。这意味着开发人员必须回头在非高峰时段重试他们的高开销改动。通过将负担转移到一个非常局限的改动和开发人员子集上，这让我们对进入队列的内容拥有了控制权。

英文原文

to

protect

the

rest

of

the

queue,

what

we

do

uh

in

this

uh

phase

is

just

to

reject

any

expensive

change

during

peak

hours.

What

this

means

is

that

the

developer

has

to

come

back

and

retry

their

expensive

change

outside

peak

hours.

Uh

so

this

gives

us

control

into

what

enters

the

queue

by

moving

the

burden

onto

a

very

subset

of

changes

and

developers.

我们来看看准入关控的权衡。在实施这一点后，我们确实观察到了吞吐量的增长。我们实现了超过 20% 的增长。在工作时间内，我们的合入率（land rate）从 130 左右上升到了约 160。但这是有代价的，代价就是开发人员变成了调度器。并不是所有开发人员都这样，因为我们确实保护了较小的改动。但你可以想象，如果你是一名拥有横跨整个代码的任意类型代码库的开发人员，你在这当中会受到特别的针对，因为你所做的工作天然就落入了这个高开销改动的分类中。所以我们在高峰时段保护了容量，但不得不让开发人员等待。下一张幻灯片。

英文原文

Uh

let's

look

at

the

trade-off

of

admission

gating.

Uh

after

we

implemented

this,

we

definitely

observed

an

increase

in

our

throughput.

We

were

able

to

achieve

over

20%

increase.

Uh

our

land

rate

went

from

130

to

about

160

during

business

hours.

But

it

came

at

a

cost

and

the

cost

is

that

developers

become

the

scheduleulers.

uh

not

all

developers

since

we

do

protect

the

smaller

changes.

But

you

can

imagine

if

you

are

a

developer

who

owns

any

types

of

libraries

uh

that

span

the

entire

code,

you

will

be

particularly

targeted

in

this

because

the

work

that

you

do

uh

naturally

falls

into

this

expensive

change

bucket.

Uh

so

we

managed

to

protect

the

capacity

during

peak

hours

but

had

to

make

developers

wait.

Uh

next

slide.

推迟队列与冲突放宽

所以我们做的下一件事，更多是为了改善准入关控带来的弊端中的开发人员体验。要是我们不去直接拒绝高开销改动，而是能够将它们记录下来，并根据队列的状态将它们重放回主队列，会怎样呢？比如在我们对容量有更好的掌控时。而这正是推迟机制所实现的。在高峰时段，如果有高开销请求进来，我们不再拒绝它，而是将其放入推迟队列中进行持久化保存。

英文原文

So

the

next

thing

that

we

do

is

targeted

more

towards

bettering

the

developer

experience

uh

of

uh

the

cons

that

come

from

admission

gating.

Uh

what

if

instead

of

outright

rejecting

expensive

changes,

we

are

able

to

record

them

and

replay

them

back

into

the

main

queue

uh

based

on

the

status

of

the

queue.

Uh

when

we

have

let's

say

a

better

uh

handle

on

the

capacity

and

that's

exactly

what

deferrals

implement

during

peak

hours.

If

an

expensive

request

comes

in,

we

no

longer

reject

it

but

put

it

into

a

deferred

queue

where

it

is

perfected.

一旦关卡允许——通常是在例如我们可以说高峰时段之后——我们就会把这个持久化保存的请求放回活跃提交队列中。所以本质上，我们是在以一种当队列有能力处理时再释放更多高开销改动的方式来整形流量。因此这解决的问题是：开发人员提交一次，然后当容量允许时，我们会自动为他们释放。下一张幻灯片。

英文原文

And

once

the

gates

permit

usually

upon

uh

for

example

we

can

say

after

peak

hours

uh

we

put

this

persisted

request

back

onto

the

active

submit

queue.

So

essentially

we

uh

shape

the

traffic

in

a

way

where

we

release

more

expensive

changes

when

the

queue

is

able

to

handle

it.

So

what

this

solves

is

a

developer

submits

once

and

then

we

automatically

release

it

for

them

when

the

capacity

allows.

Next

slide.

这里的权衡在于，它为开发人员解决了问题。他们不用早起、熬夜、试图找一个队列情况看起来良好的特定时间窗口然后在自己那边点击按钮。我们会为他们处理好这一点。但它的另一面，伴随而来的坏处是，它并没有真正解决开发人员的等待问题。你仍然需要等待同样长的时间，高开销改动才能进入队列。下一张幻灯片。

英文原文

So

the

trade-off

here

is

it

solves

the

problem

for

the

developer.

They

don't

have

to

wake

up

early,

stay

up

late,

try

to

find

a

particular

window

where

the

queue

looks

good

and

hit

the

button

on

their

end.

We

will

handle

that

for

them.

But

the

other

side

of

this,

the

bad

that

comes

with

this

with

this

is

that

it

doesn't

really

solve

the

waiting

for

the

developer.

You

still

have

to

wait

the

same

amount

of

time

for

your

expensive

change

to

go

in

to

the

queue.

Uh

next

slide.

所以在我们看下一个也就是我们所做的最后一项优化之前，我想稍微提一下什么是目标重叠（target overlap）的概念，以及它是如何在提交队列中将工作串行化的。回想一下 Freedom 提到的我们所执行的目标分析，用来确定两个请求是应该串行化还是可以并行处理。

英文原文

So

before

we

look

at

the

next

uh

and

final

optimization

that

we

did,

uh

I

want

to

touch

a

little

bit

about

the

uh

on

the

concept

of

what

a

target

overlap

is.

uh

and

how

that

serializes

work

in

sub.

Um

so

recall

that

freedom

mentioned

the

target

analysis

that

we

perform

uh

to

determine

whether

two

requests

should

be

serialized

or

can

they

go

in

parallel.

我们这里看的一个例子是一个严格单连通分量（strict one connected component）。这意味着 A 到 F 这些请求中的每一个都触及目标的某个子集，并且每对视图之间至少存在一个目标重叠。比如 A 和 B 至少在一个目标上重叠，B 和 C 至少在一个目标上重叠。C、D、E 和 F 同样如此。这使得提交队列把所有这些请求归为单个分量。这意味着所有这些都必须一起处理。而这基本上就是阻碍我们更快开展工作的原因。

英文原文

So

an

example

that

we

are

looking

at

here

uh

is

a

strict

one

connected

component.

What

this

means

is

that

each

of

these

requests

A

through

F

touch

some

subset

of

targets

and

all

of

there's

at

least

one

overlap

of

targets

between

each

pair

of

views.

So

like

A

and

B

overlap

on

at

least

one

target,

B

and

C

overlap

at

least

one

target.

Similarly

for

C,

D,

E,

and

F.

And

what

this

makes

uh

submit

Q

do

is

group

all

of

these

requests

into

a

single

component.

And

that

means

that

all

of

these

have

to

go

together.

And

this

is

basically

what

prevents

uh

us

from

doing

work

faster.

我们做的下一项优化就是：我们如何打破这种局面？有没有一种方法可以让我们聪明地将其拆解开来？下一张幻灯片。冲突放宽正是帮助我们以知情的方式打破这些连接的技术。在之前基本上就是现状：两个请求之间只要有任何相关的重叠，就会让它们彼此冲突。

英文原文

Uh

and

the

next

optimization

we

do

is

how

do

we

break

this?

Is

there

a

way

where

we

can

break

this

up

in

a

smart

way?

Um

next

slide.

So

conflict

relaxation

is

what

helps

us

break

these

links

uh

in

an

informed

fashion.

uh

before

is

basically

the

status

quo.

Any

relevant

overlap

between

two

requests

uh

makes

them

conflict

with

each

other.

我们的下一场演讲……将在大约……后开始

英文原文

>> Our

next

talk

judgment

is

the

last

job

will

begin

in

approximately

通过放宽，我们基本上执行三个步骤。首先，我们对重叠程度进行排序。所以这不再是一个“这两个请求之间是否存在至少一个重叠”的二元决定。更多地把它看作是：这种重叠有多强？一旦我们掌握了这一点，我们就会保留更强的连接，并根据我们设置的配置值放宽较弱的连接，以重新换取并行度。下一张幻灯片。

英文原文

>> with

relaxation.

We

essentially

do

three

steps.

First,

we

rank

the

overlap.

So

it's

no

longer

a

binary

decision

of

is

there

at

least

one

overlap

between

these

two

requests.

Think

of

it

more

as

a

how

strong

is

this

overlap.

Uh

once

we

have

that

we

keep

the

stronger

links

and

relax

the

weaker

ones

to

buy

back

parallelism

based

on

a

config

value

that

we

set.

Um

next

slide.

所以第一步基本上是对这些边或者说这些重叠进行打分。我们使用 Jaccard 打分来计算这种依赖重叠。我们基本上有重叠的目标，我们会对此进行检查，并检查引入的任何新依赖边。我们将这两者组合计算成一个介于 0 和 1 之间的连续数值分数，其中分数越高表示重叠越强，分数越低表示边越弱、两者之间重叠的目标越少。下一张幻灯片。

英文原文

So

the

first

step

is

essentially

scoring

these

edges

uh

or

these

overlaps.

We

use

a

jakard

scoring

uh

for

computing

this

dependency

overlap.

We

basically

have

overlapping

targets.

Uh

we

check

for

that

and

any

new

dependency

edges

that

are

introduced.

We

compute

a

combination

of

these

two

into

a

continuous

valued

score

between

0

and

one

where

a

higher

value

of

score

indicates

a

stronger

overlap

and

a

lower

value

of

the

score

indicates

a

weaker

edge

with

fewer

targets

that

overlap

between

this.

Next

slide.

一旦掌握了这些信息，我们基本上就会重新构建这个冲突图。对吧？沿用同样的例子，并为早先幻灯片中的冲突加上一些数值。假设我们有这六个改动，并且我们设定的配置值表示我们希望队列中始终保持至少 50% 的并行度。这意味着我们至少需要三个彼此独立工作的独立队列，并且基于我们计算出的重叠分数，我们能够做出丢弃某些边的选择，以便能够达到我们设定的目标队列规模。

英文原文

Once

we

are

armed

with

this

knowledge,

we

basically

uh

make

this

conflict

graph

again.

Right?

So

taking

the

same

example

and

adding

some

numbers

to

the

uh

conflicts

that

we

have

uh

from

the

earlier

slide.

Let's

say

we

have

these

six

changes

and

the

config

value

we

set

says

we

want

at

least

50%

parallelism

in

the

queue

at

all

times.

What

this

means

is

that

we

need

at

least

three

separate

cues

that

are

working

uh

independent

of

each

other

and

based

on

the

uh

overlap

scores

that

we

have

computed

we

are

able

to

make

this

choice

of

dropping

certain

edges

in

order

to

be

able

to

achieve

that

target

uh

Q

size

that

we

have.

So

所以在这种情况下，最弱的边位于 B 和 C 以及 B 和 E 之间，我们丢弃了这两条边，这为我们带来了三个独立的连通分量。这就是我们基本上如何从六个请求的串行链条转变为并行链条，其中每两个请求分别存在于三个独立的队列中。

英文原文

in

this

case

uh

the

weakest

edges

are

between

B

and

C

and

B

and

E

and

we

drop

those

two

which

gives

us

three

separate

connected

components.

Uh

and

this

is

how

we

essentially

go

from

a

serial

chain

of

six

requests

to

a

parallelized

chain

where

two

each

exist

in

three

separate

cues.

下一张幻灯片。那么接下来就是一个问题：我们究竟在什么地方做丢弃这些边的决定，对吧？有两个地方我们可以做这件事。一个是在全局层面上，每次我们尝试分配并构建配置图时，我们基本上每次都可以重新计算这个。而这所导致的是，它会向图中引入一定程度的不稳定性，这种不稳定性取决于进入的任何新请求

英文原文

Next

slide.

So

next

comes

a

question

of

um

where

exactly

do

we

make

this

determination

of

uh

dropping

the

edges

right?

So

there

are

two

places

where

we

can

do

this.

Uh

one

is

at

a

global

level

every

time

uh

we

are

trying

to

assign

uh

config

graphs

and

building

them.

Uh

we

can

basically

recomputee

this

every

time.

And

what

that

does

is

it

introduces

some

level

of

instability

into

the

graph

which

is

determined

by

any

new

requests

that

enter

队列，任何完成并退出队列的工作。所以，呃，本质上我们可以那么做，呃，但我们选择采取的做法相反，是对重叠进行更局部、呃、局部的分配。

英文原文

the

queue,

any

work

that

finishes

and

exits

the

queue.

So

uh

essentially

what

we

we

can

do

that

uh

but

what

we

chose

to

do

instead

was

a

more

uh

local

assignment

uh

of

overlaps.

我们选择只在有新请求到达的检查点执行这一操作。因此，一旦有新请求进来，我们基本上就拿它与队列中现存的所有事物计算边，做出决策，然后就这样保持不变。下一次决策只会在又一个新请求到达时发生。所以概念很简单：我们在进入时分配一次，呃，基本上就复用它。

英文原文

And

the

place

that

we

choose

to

do

this

at

is

only

at

the

new

arrival

uh

checkpoint.

So

once

a

new

request

comes

in,

we

essentially

take

that

and

compute

the

edges

of

this

with

everything

that

exists

in

the

queue,

make

the

decisions

and

that's

how

it

stays.

The

next

decision

making

will

only

happen

when

another

new

arrival

comes

in.

So

the

concept

is

simple.

We

assign

once

at

entry

time.

uh

and

basically

reuse

this.

嗯，这帮我们做到的，就是根据我们之前创建的冲突图——冲突图所调度的任何工作都会被保留，呃，它能帮我们保留投机构建，从而帮助我们更快地、呃、走向下一步的最终确定。

英文原文

Um

what

this

helps

us

do

is

any

work

that

we

schedule

based

on

previous

conflicts

gra

conflict

graphs

that

we

have

created

is

preserved

uh

and

it

helps

us

hang

on

to

speculative

builds

that

can

help

us

move

faster

uh

towards

a

finalization

of

its

next.

所以，这项最终干预措施带来的权衡是我们见过的最大的。呃，我们看到了超过 50% 的吞吐量增长。我们从大约每小时 160 行增长到了将近 250 行，在高峰时段甚至更高，呃。所以它确实有效，不仅帮助我们、呃、显著提高了吞吐量，同时还给我们带来了、呃、极好的 P95 着陆时间，但它也是有代价的，就像所有事情一样；我们在这里做出的权衡，基本上就是 PTO 在第一张幻灯片中提到的内容。

英文原文

So

the

tradeoff

of

this

final

um

intervention

was

the

biggest

that

we

saw.

Uh

we

saw

over

50%

increase

in

the

throughput.

So

we

went

from

about

160

lines

per

hour

to

almost

250

sometimes

even

greater

uh

during

peak

hours.

Uh

so

it

worked

in

that

it

not

only

helped

us

uh

significantly

improve

our

throughputs

while

giving

us

uh

extremely

good

uh

P95

land

tanks

but

it

came

at

a

cost

as

everything

does

and

the

trade-off

that

we

have

given

away

here

is

basically

what

PTO

alluded

to

in

the

very

first

slide.

Heat。Heat。

英文原文

Heat.

Heat.

Heat。

英文原文

Heat.

Heat。

英文原文

Heat.

我。嘿。嘿。

英文原文

me.

Hey. Hey.

Heat。Heat。

英文原文

Heat. Heat.

Heat。Heat。

英文原文

Heat.

Heat.

产品构建的未来与判断力

大家好，我是 Ean，在 Google DeepMind 负责 anti-gravity。今天我要谈谈产品构建的未来、判断力是什么样的，以及我们在 Google 发布 anti-gravity 过程中学到的一些教训。

英文原文

Hey

everyone,

I'm

Ean

and

I

work

on

anti-gravity

at

Google

Deep

Mind.

Today

I'm

going

to

talk

about

the

future

of

building

products,

what

judgment

looks

like,

and

some

lessons

that

we've

learned

from

shipping

anti-gravity

at

Google.

所以，anti-gravity 是一个借助智能体完成工作的工具，我们正在努力为开发者和知识工作者打造最好的界面，来完成高智力任务。

英文原文

So

anti-gravity

is

a

tool

to

get

work

done

with

agents

and

we're

trying

to

make

the

best

interface

for

developers

and

knowledge

workers

to

do

high

intelligence

tasks.

我们过去所认识的软件工程已经彻底改变了。我们现在能做的事情，在一年前连做梦都想不到。

英文原文

The

software

engineering

we

used

to

know

has

completely

changed.

We

can

do

things

now

that

we

wouldn't

have

even

dreamed

of

a

year

ago.

一句话就能变成一个操作系统，而我的声音可以创造一款 3A 游戏。

英文原文

A

sentence

can

turn

into

an

operating

system

and

my

voice

can

create

a

AAA

game.

那么，留给我们的还剩下什么？

英文原文

So,

what's

left

for

us?

我们使用的工具已经变了。我们过去常常使用代码编辑器，它已经存在了几十年。但现在，就算不是更频繁，也是每年都会出现一种如何开展工作的新范式。它重新定义了我们的工作究竟是什么样子的。

英文原文

The

tools

we

use

have

changed.

We

used

to

use

the

code

editor.

It's

been

around

for

decades.

But

now,

every

year,

if

not

more

often

than

that,

there's

a

new

paradigm

for

how

to

do

our

work.

It

redefineses

what

our

work

even

looks

like.

而在构建产品这一侧，情况同样发生了改变。

英文原文

And

on

the

side

of

building

products,

that's

changed

as

well.

我们过去常常在前期花大量时间去琢磨要做什么是对的，因为构建的成本很高。它需要耗费大量的时间、精力和资源。因此，我们在那个阶段会进行更多构思并投入更多。现在这完全翻转过来了，虽然你现在仍应该理解你要解决的问题以及如何着手去做，但如果你想做点什么，它马上就能成真。

英文原文

We

used

to

spend

a

lot

of

time

upfront

figuring

out

the

right

thing

to

make

because

building

was

expensive.

It

took

a

lot

of

time,

effort,

and

resources.

So,

we

ideated

more

and

invested

more

in

that

phase.

This

is

completely

flipped

where

now

you

should

still

understand

the

problem

you're

trying

to

solve

and

how

to

go

about

it.

But

if

you

want

to

make

something,

it

can

exist

just

like

that.

你可以尝试它，感受它，获取反馈，并理解那些你以前可能没有意识到的细微差别，然后用它来指导如何向前推进。

英文原文

You

can

try

it,

feel

it,

get

feedback,

and

understand

the

nuances

that

you

might

not

have

realized

about

it

before

and

use

that

to

inform

how

to

move

forward.

这很令人兴奋，但也有点让人产生错觉。关闭大脑、盲目接受为我们做好的东西，现在比以往任何时候都更容易。所以，如果任何人都能做出任何东西，那么知道到底做什么是对的，就比以往任何时候都更重要。

英文原文

This

is

exciting,

but

it's

also

a

little

bit

deceptive.

It's

easier

than

ever

to

just

turn

our

brains

off

and

blindly

accept

what's

made

for

us.

So,

if

anything

can

be

made

by

anyone,

then

it's

more

important

than

ever

to

know

what

the

right

thing

to

even

make

is.

每个人对于品味或判断力是什么样都有自己的定义。我认为这没问题。这很好，因为我们的工作方式各不相同，我们努力追求的目标也各不相同。

英文原文

Everyone

knows

their

own

definition

for

what

taste

or

judgment

looks

like.

And

I

think

that's

okay.

That's

good

because

we

all

work

in

different

ways

and

we're

all

working

towards

different

things.

但在这次演讲结束时，我希望能帮助大家对判断力和产品构建对你而言是什么样子形成一个更清晰的概念。

英文原文

But

by

the

end

of

this

talk,

I'm

hoping

that

I

can

help

you

form

a

little

bit

of

a

clear

idea

on

what

judgment

and

building

products

looks

like

to

you.

我经历过初创公司的规模，到现在进入了 Google DeepMind，这里的规模让人感觉相当不可思议。我们正在打造前沿模型，而在产品端，我们能够利用这一内部前沿，去构建承载大家即将看到的下一步智能发展的界面。

英文原文

I've

gone

from

the

scale

of

working

at

startups

to

now

at

Google

DeepMind

where

the

scale

is

it

feels

pretty

incredible.

We're

making

frontier

models

and

on

the

product

side

we're

able

to

use

this

internal

frontier

to

build

the

interfaces

that

will

use

the

next

steps

everyone

will

see

in

intelligence.

我们离研究非常近，我们可以非常快地进行实验和迭代，并交付给 Google 内部超过 100,000 人使用。

英文原文

We're

very

close

to

the

research

and

we

can

experiment

and

iterate

really

fast

and

ship

to

over

a

100,000

people

inside

Google.

Google 的这些员工正在内部以非常宏大且独特的方式使用 anti-gravity。这创造了一个非常酷的反馈回路，让我们能够继续构建并做些了不起的事情。

英文原文

These

people

at

Google

are

using

anti-gravity

internally

in

really

ambitious

and

unique

ways.

And

this

creates

a

really

cool

feedback

loop

where

we

can

continue

building

and

do

awesome

things.

我想带大家梳理一下我们从这个循环中学到的三大经验教训。

英文原文

I

want

to

walk

you

through

three

big

lessons

that

we've

learned

from

this

cycle.

工具随智能而扩展

所以我想我们大家都见过第一种模式：我们使用的工具随着智能而向上扩展。

英文原文

So

I

think

we've

all

seen

this

first

pattern

of

the

tools

we

use

scaling

up

with

intelligence.

我们过去以一种相对非常直接的方式构建产品：发现一个问题，深入理解它，然后带着明确的目标做出人们想要的东西。现在，那个目标在变动。能做什么的上限正在提高。随之而来的是，你在所构建产品中能创造的潜在价值也在发生改变。

英文原文

We

used

to

build

products

in

a

pretty

relatively

straightforward

way.

We

identify

a

problem,

understand

it

deeply,

and

make

something

that

people

want

with

a

clear

target.

Now

that

target

is

moving.

The

ceiling

is

going

up

for

what's

even

possible

to

make.

And

with

that,

the

possible

value

that

you

can

create

in

the

products

that

you

build

is

changing

as

well.

因此，每当我工作时，我总会回想几个念头：我们现在能实现什么，以及在刚好超出我们能实现的边界之外又有什么。我们在 DeepMind 的构建过程中看到了两种主要的呈现方式。

英文原文

So

I'm

always

coming

back

to

a

few

thoughts

whenever

I'm

working

of

what

we

can

achieve

now

and

what's

just

past

the

edge

of

what

we

can

achieve.

And

we've

seen

this

look

like

two

main

ways

in

building

a

deep

mind.

今年早些时候随着我们的模型变得更好，它们在彼此通信和编排方面变得更强了。我们创建了团队协作系统，以开展更具抱负的工作。我们提升了智能水平，让用户能够做更多的事情。在当时，这非常棒。你可以创建操作系统以及我们以前从未见过的非常酷的东西，但它也是相当规定性的，因为这是当时模型所需要的。更近期一些，这呈现为我们版本的智能体集群（agent swarms）。你现在实际上可以在 anti-gravity 中体验这个，但这已经在 Google 内部为我们创造了大量的价值。人们可以更有野心。有更多的智能体在一起协同工作，我们得以放宽约束来实现更多目标。

英文原文

As

our

models

got

better

earlier

in

the

year,

they

got

better

at

communicating

with

and

orchestrating

each

other.

We

created

systems

for

teamwork,

for

to

do

more

ambitious

work.

We

stepped

up

in

intelligence

to

let

users

do

more.

At

the

time,

it

was

awesome.

You

could

create

operating

systems

and

really

cool

things

we've

never

seen

before,

but

it

was

also

pretty

prescriptive

because

the

models

needed

at

the

time.

More

recently,

this

has

looked

like

our

version

of

agent

swarms.

You

can

actually

try

this

now

in

anti-gravity,

but

this

has

been

creating

a

ton

of

value

inside

Google

for

us.

People

can

be

more

ambitious.

There

's

more

agents

working

together

and

we're

able

to

relax

the

constraints

to

achieve

more.

你可以确信，随着接下来的模型推出，我们将看到更酷的模式和我们未曾预料到的东西。

英文原文

You

can

bet

that

with

the

next

models

coming

out,

we'll

see

even

more

cool

patterns

and

things

we

didn't

expect.

但我想我们大家都已经有点知道这一点了。过去几年我们一直在目睹这一点：我们使用的所有工具都只是在不断增加越来越多样的新鲜、炫酷方式，来随着智能向上扩展。

英文原文

But

I

think

we

all

already

kind

of

knew

this.

We've

been

seeing

this

for

the

past

few

years

where

all

the

tools

we're

using

are

just

adding

more

and

more

new

cool

ways

to

scale

up

with

intelligence.

但我见到的随智能演进的第二种方式就有点更微妙了，因为有野心并不总是意味着做加法。

英文原文

But

the

second

way

I've

seen

evolving

with

intelligence

look

is

a

little

bit

trickier

because

being

ambitious

does

not

always

mean

adding

more.

打破既有假设

让我们把时间往回倒退一点，一起回顾产品中的一个决策。

英文原文

Let's

go

back

in

time

a

little

bit

and

walk

together

through

a

decision

in

the

product.

我们过去使用 AI 工具的方式非常类似于像这样结构化的回合制格式。我会请求一些小范围的改动，比如编辑几个文件。模型去完成，然后我们一步一步转向下一件事。在当时，能够排队指令是有意义的，这样我想要的后续事情就可以逐一传递给模型。它们可能还没那么聪明，无法一次性理解所有事情，但逐步给它有限的上下文是有意义的，这样它就能够，你知道的，一点一点把工作搞定。

英文原文

The

way

we

used

to

use

AI

tools

looked

very

much

like

a

structured

turn-based

format

like

this.

I

would

ask

for

some

small

scoped

change.

For

example,

editing

a

few

files.

The

model

would

do

it

and

then

we

move

on

to

the

next

thing

step

by

step.

It

made

sense

at

the

time

to

be

able

to

cue

instructions

so

the

next

things

I

wanted

could

be

delivered

to

the

model

one

at

a

time.

They

might

not

have

been

smart

enough

to

get

everything

at

once,

but

it

made

sense

to

give

it

limited

context

progressively

so

it

could,

you

know,

get

the

work

done

bit

by

bit.

快进到现在。现在发生的事情多得多了：长时间运行的智能体、异步任务、外部触发器，以及我们正在做的所有其他事情。事情看起来更加复杂，而智能体也变得更聪明了。所以，也许我们过去做出的决定可能不再正确了。也许更好的模式是更多地将智能体视为一个能够做出更好决策的实体，并立刻向它传达用户意图，就像这样。

英文原文

Fast

forward

to

now.

There's

a

lot

more

going

on.

Longunning

agents,

asynchronous

tasks,

external

triggers,

and

everything

else

we're

doing.

Things

are

looking

more

complicated,

and

the

agents

are

also

smarter.

So

maybe

the

decision

we

made

in

the

past

might

no

longer

be

correct.

Maybe

a

better

pattern

looks

like

treating

the

agent

more

as

an

entity

that

can

make

better

decisions

and

giving

it

user

intent

immediately,

just

like

this.

也许它会选择将任务编排给一群工作智能体，或者在我们手头的当前任务完成后再去执行该任务。它更聪明了，所以它可以自己做决定。

英文原文

Maybe

it'll

choose

to

orchestrate

to

a

bunch

of

worker

agents

or

do

the

task

after

we're

done

the

current

task

at

hand.

It's

smarter,

so

it

can

decide

for

itself.

随着智能的发展，我们过去做出的决定很容易过时，你需要挑战自己原有的假设。

英文原文

The

decision

we

made

in

the

past

can

easily

become

obsolete

as

intelligence

grows,

and

you

need

to

challenge

your

prior

assumptions.

如果我们把我们所能实现的成果时间线分成三个明确的部分：如果你针对过去构建得太远，它就会过时；如果你为未来构建得太远，它可能会很有远见但脱离实际。

英文原文

If

we

separate

the

timeline

of

what

we

can

achieve

into

three

clear

parts,

if

you

build

too

far

in

the

past,

it'll

be

obsolete.

If

you

build

too

far

for

the

future,

it

could

be

visionary

but

impractical.

对我们来说，快速构思并努力在正确的时间窗口内进行构建以创造最大的价值，这是很重要的。挑战你过去的假设，重建你的基础，以创建能随智能扩展的良好系统。

英文原文

It's

important

for

us

to

ideate

quickly

and

work

towards

building

in

the

right

time

window

to

create

the

most

value.

Challenge

your

past

assumptions

and

rebuild

your

foundations

to

create

good

systems

that

scale

with

intelligence.

有了这些基础，我们在 Google 看到又一种非常棒的模式演化或显现出来，即我们看到了打造一个承载雄心的平台的价值。

英文原文

With

these

foundations,

we've

seen

another

really

awesome

pattern

evolve

or

come

out

at

Google

where

we

see

the

value

in

creating

a

platform

for

ambition.

类似这样的事情听起来可能会很耳熟。我过去常常以一种非常手动的形式使用设计工具：我会手动创建画板、构建用户界面，亲手制作东西。而那个工具，如果我们看看它如何

英文原文

So

something

like

this

might

sound

familiar.

I

used

to

use

design

tools

in

a

very

manual

way.

I'd

create

frames,

build

user

interfaces

manually,

and

create

things

by

hand.

And

the

tool,

if

we

look

at

how

从过去到现在，它发生的变化看起来非常相似。但我使用它的方式已经彻底改变了。也许现在智能体会为我生成成百上千个可视化图表，供我在深入探究某件事之前先做非常宽泛的浏览。

英文原文

it changed from before to now, it looks pretty similar. But the way I use it has completely flipped. Maybe now agents are creating hundreds of visualizations for me to go over very broadly before I go deeper into something.

现在构建软件并不总是意味着以一种非常明确且确定性的方式向用户提供价值。用户可以创造自己的杠杆，并为他们自己的雄心留出空间。让我带大家了解一下这对我们而言是怎样的。

英文原文

Building software now doesn't always look like giving the user value in a very clear and deterministic way. Users can create their own leverage and make room for their own ambition. Let me walk you through how this has looked for us.

在我们的产品中，有很大的可扩展空间。人们可以构建适合自己的专门工具，而这通常最终又会反哺到产品本身中。我来给大家展示一些例子。比如可能会出现个人软件，我可以在那里做一个仪表盘，智能体会跨反馈渠道和错误修复进行扫描，以完全符合我喜好的方式向我展示所有这些信息。

英文原文

In our product, there's a lot of room for extensibility. People can build specialized tools that work for them and this can often end up leading back into the product. I'll show you some examples. Personal software can come up where I can make a dashboard where agents will scan across feedback channels and bug fixes to show me all this information in a way that just I like.

也许它会呈现为面向我团队的软件，设计师可以创建专门适合他们的工具，在其中就演示进行视觉上的协作。

英文原文

Maybe it'll look like software for my team where designers can create the tool that works just for them where they can collaborate visually on demos.

这还可以进一步延伸拓展。Google 的科学团队正在利用他们的科学插件做一些非常棒的事情。

英文原文

This can also branch out even more. The science team at Google is doing some awesome things with their science plugins.

我们正在为广大用户打造这样一个承载雄心的平台，让他们创造自己的价值并建立自己的杠杆。而这实际上可以反哺到你正在打造的核心产品中。

英文原文

We're creating this platform for ambition for a wide range of users to create their own value and create their own leverage. And this can actually feed back into the product you're making at its core.

我之前提到的智能体表单例子，最初其实是通过我们的可扩展性系统构建的一个插件。它在 Google 内部为我们创造了巨大的价值。通过为用户提供探索和为自己创造价值的表面空间，我们得以从中学习并在产品中进行迭代，使核心变得更好。

英文原文

The agents form example I mentioned earlier actually started as a plug-in through our extensibility system. It's created a ton of value internally for us at Google. And by giving the users surface area to explore and create value for themselves, we were able to learn from it and iterate in the product to make the core better.

让你的用户心怀抱负，并见证这种抱负反哺到你的产品中，从而为大家创造更多的价值。

英文原文

Let your users be ambitious and watch it feedback into your product to to create more value for everyone.

人类判断与软件工程的核心

我想讲的最后一个原则，可以说是适用于所有这些内容的。

英文原文

The last principle I want to go over kind of applies to all of this.

本次大会上有很多关于软件工厂的精彩演讲。大家知道，这确实非常令人兴奋。我们可以为我们的智能体提供构建更多内容、更可靠地进行验证以及更快发布的方法。这很令人振奋。但是，如果以往软件工程的所有构成部分都被抽象化了，那么我们该如何定义人类判断所剩余的价值呢？我认为人类判断比以往任何时候都更加重要。如果我让一个智能体去“把这张幻灯片做好一点”，也许它会添加一些可视化图表、一堆仪表盘、炫酷的色彩等等。也许它会用 Rust 重写整个幻灯片展示程序。

英文原文

There's a lot of great talks at this conference about software factories. You know, it's really exciting. We can give our agents ways to build more, verify more reliably, and ship faster. It's exciting. But if all the parts of what software engineering used to look like get abstracted away, then how do we define what the remaining value in our human judgment is? I'll argue that human judgment is more important than ever. If I ask an agent to say, make this slide better, maybe it'll add some visualizations and a bunch of dashboards and cool colors and everything. Maybe it'll rewrite the whole slideshow in Rust.

所以，它会解决每一个问题。但每一个问题都需要被解决吗？

英文原文

So, it'll solve every problem. But does every problem need to be solved?

以大约一年前 Anti-gravity 刚推出时智能体管理器的样子为例。如果我们想在智能体看来让它变得更好，那可能意味着添加更多细节、发布所有东西、在智能体采取的每一个步骤中加入更多细节，并为用户提供控制一切的旋钮。

英文原文

Take for instance what the agent manager looked like around a year ago when anti-gravity first launched if we want to make it better to an agent. That might look like adding a lot more detail, ship everything, add more detail into every single step the agent is taking and give the user knobs for everything

这在某些方面可能更好。但这是盲目的。我们很容易盲目接受摆在我们面前的第一个方案。对我们而言，“更好”究竟意味着什么？如果我们试图打造一个更好的智能体管理器界面，也许那看起来会有所不同。也许我们需要快速发布，但在迭代过程的每一步都要理解对我们而言“更好”究竟意味着什么，以及下一个方向是什么。

英文原文

might be better in some ways. But it's blind. It's easy to blindly accept the first thing that we're given. What does better even mean to us? If we're trying to make a better agent manager interface, maybe that looks a little bit different. Maybe we need to ship fast, but understand at each step of the iteration process what better actually means to us and what the next direction is.

让我们从这里开始。

英文原文

Let's start here.

更好可能意味着简化界面，在语义上只向用户准确展示智能体当前正在做的最新事情，这样他们就不会面对满屏可能根本无关紧要的文字和工具调用。也许它看起来像是简化整个界面以减轻用户的认知负担，从而使他们能够更轻松地进行多任务处理，并能够以不同的方式实现更多目标。

英文原文

Better might mean simplifying the interface, only showing the user exactly what the latest thing the agent is doing semantically so they're not shown a wall of text and tool calls that might not actually matter. Maybe it looks like simplifying the whole interface to give the user less cognitive load so that they can multitask easier and be able to achieve more in a different way.

因此，这让我们更接近 Anti-gravity 今天的样子。这一过程中的所有这些步骤，都让我们更接近“优秀”的含义。

英文原文

So this kind of brings us closer to what anti-gravity looks like today. All these steps along the process have brought us closer to what good means.

而且我们正在快速进行大量改进。因此，我非常期待大家能看到它。

英文原文

And we're making a lot of improvements fast. So I'm excited you all for you all to see it.

现在正是尝试多种事物的时候。要有雄心。现在比以往任何时候都更令人振奋。而且随着模型变得越来越好，实际上还会变得更加令人兴奋。

英文原文

Now is the time to try many things. Be ambitious. It's more exciting than ever. And it'll actually get more exciting as the models get better.

尝试许多方向，但花点时间思考。运用你的人类判断力。仅仅因为你率先完成了某件事的创建（这在瞬间就能完成），并不意味着你真正完成了。

英文原文

Try many directions, but take a moment to think. Use your human judgment. Just because you're finished creating something first, which can be done in an instant, it doesn't actually mean you're done.

思考一下你真正想要实现的是什么，以及你正在制作的东西给人什么样的感受。

英文原文

Think about what you're actually trying to achieve and how the thing you're making feels.

快速迭代。确定你可以走的许多方向并尝试它们，但要找到正确的事物并深入钻研进去。

英文原文

Iterate quickly. Identify many directions you can go in and try them, but arrive at the right thing and go deep into that.

因此，要留出思考的时间，留出思考的空间，并运用你的人类判断力。

英文原文

So make time to think, make room to think, and use your human judgment.

总结一下，我们学到的一些经验包括：随着智能的扩展，这并不总是意味着添加更多内容。它可能意味着重塑你的基础。依托这些基础，让你的用户以你之前可能未曾预料到的新方式去怀抱雄心。在所有这一切的最后，当我们发布得越来越快时，留出思考的空间，守护软件工程中属于人性的那一面。

英文原文

To summarize, some of the lessons we've learned involve if scaling up with intelligence, which doesn't always mean adding more. It could mean rebuilding your foundations. Take these foundations and enable your users to be ambitious in new ways you might not have expected before. And at the end of all this, when we're shipping faster and faster, make room to think and protect the human side of software engineering.

在 Google，通过打造一个承载雄心的平台，这带来了一些非常棒的成果：对 YouTube 的改进、更好的视频解码器以及内存优化。

英文原文

At Google, this has led to some awesome things by creating a platform for ambition, improvements to YouTube, better video decoders, and memory optimizations.

至于 DeepMind 和 Anti-gravity，随着新模型的推出，这是一个比以往任何时候都更加令人兴奋的时刻。

英文原文

As for DeepMind and Anti-gravity, it's a more exciting time than ever with new models coming out.

在产品方面，我们的推进速度也比以往任何时候都要快。这些只是我们在过去两周内宣布的一些功能发布。

英文原文

And on the product side, we're also moving quicker than ever. These are just some of the feature launches we've announced in the last two weeks.

我们正在继续利用今天涵盖的这些原则，致力于构建面向更高智能的未来界面。我非常期待大家看到它。最后我想把这句话留给大家：软件工程的这个新时代实际上值得庆祝。智能体所做的事情可能越来越多，但解决真实问题并运用良好的产品思维也比以往任何时候都更重要。保护好你的判断力，去打造人们真正想要的东西。

英文原文

And we're continuing to use the principles we've covered today to work on the future of interfaces for higher intelligence. I'm excited for you all to see it. And I'll leave you with this. This new age of software engineering is actually worth celebrating. Agents might be doing more and more, but it's also more important than ever to solve real problems and use good product thinking. Protect your judgment and build things that people want.

谢谢大家。

[掌声]

英文原文

Thank you.

[applause]

演讲交接与软件工厂介绍

Hey, hey, hey. Heat. Heat. Hey. Heat. Heat. N. Heat. Heat. every Heat. Heat. Heat. Heat. Heat. Hey, Heat. Heat. Heat. Heat. Heat. Heat. Hey, Heat.

英文原文

Hey, hey, hey.

Heat. Heat.

Hey.

Heat. Heat. N.

Heat. Heat.

every

Heat.

Heat.

Heat. Heat.

Heat. Hey, Heat.

Heat.

Heat.

Heat. Heat.

Heat.

Hey, Heat.

好的，各位。希望一切设备都开启正常。大家可以看到这里的棕地工厂。好的。我叫 Justin McCarthy。我是 Diffusion 的创始人兼 CEO。我们正在将模型的能力扩散到全球经济中。外面的世界很大，体量有百万亿美元。我们有八十亿人。所以有很多事情要做。嗯，这就是我们正在做的事情。而我们践行的一种方式就是构建软件工厂。

英文原文

All right, folks. Hopefully everything's on. You can see the brownfield factory here. All right. My name is Justin McCarthy. I'm the founder and CEO at Diffusion. And we are diffusing the capabilities of the model into the global economy. Uh it's a big place out there. There's hundred trillion dollars of it. There's eight billion of us. So there's a lot to do. Um uh and that's that's what we're doing. And one way we're doing it is by building software factories.

嗯，实际上这次演讲，大家知道，我想它的名称里带有“工厂”这个词。嗯，这不会是非常技术性的内容。最后有一个二维码，会为大家提供一些技术性的资料。因此，我鼓励大家去获取，或者让你们的智能体去获取。我最近经常发送双通道消息，在上面这一层说面向人类的内容，然后在下面那层附上供智能体使用的内容。所以，这次演讲有点像这样。我将向大家讲述人类层面的内容，然后你们的智能体会在此次演讲结束时获取智能体通道的内容。好的。

英文原文

Um, this talk actually, you know, uh, I don't I guess it's got the factory thing in the name. Um, uh, this it's not going to be super technical. There's a QR code at the end that is going to have some technical stuff for you. So, I would encourage you to pick up or have your agents pick things up. I'm sending a lot of dual channel messages these days where I say the human stuff up here and then I attach something for the agent down there. So, that's kind of this presentation. I'm going to say human things to you and then your agents are going to pick up the agent channel stuff at the end. Um, okay.

那么，对我来说，这始于去年七月。我们当时使用的是 Claude 4 系列模型，大家知道，我很荣幸能够非常非常深入地钻研，真正探究当你开始让这些东西形成循环时会发生什么。

英文原文

So uh so this started for me uh in July of last year. Uh we were on the Clawude 4 series of models and um you know I had the privilege of being able to dive very very deep um uh and really ask what happens uh when you begin looping these things.

这就是我和团队共同构建的软件工厂开端的故事。我想说，大家知道，即使在 4 系列模型上，甚至在我们等待 GPT-5 和最初的 Codex 架构测试平台期间，大家知道，即使在那时我们也能看到这套机制即将可行的曙光，但伴随着巨大的 token 开销，而且只有在存在一个真正的鲁布·戈德堡机械（过度复杂的装置）时才行。好的。所以只有当这些循环非常勉强地拼凑在一起、并且非常依赖手工打造时，你才能看到某种输出的曙光，这种输出能够稳定输入，从而带来我们最终想要的那种可靠性。

英文原文

Um so that uh was the story of the beginning of the software factory that I built um together with uh the team and I'll say that you know even on the 4 series of models uh even as we were awaiting GPT5 and the initial codeex harness um you know even then we could see glimmers that this thing was going to work but at tremendous token expense and only in the presence of a real Rube Goldberg contraption. Okay. So only when the loops were very duct taped together um and and very sort of um uh handcrafted uh would you be able to see these glimmers of some kind of an output that would stabilize the input that would lead to the kinds of reliability um that we would ultimately want.

好的。所以我要说的是，那种感觉，如果你从本次大会上出现的关于软件工厂的演讲数量来判断，那种感觉我认为就是我们现在所知的软件工厂，而且在时代思潮中似乎已形成某种共识，即至少有这其中的一部分现在就这样自然而然地发生了，并且它需要我们投入的创造力和输入大幅减少，以

英文原文

Okay. So I'll just say that feeling um if you judge from the number of software factory talks that have been present here um at the conference that feeling um I think is what we know now as software factories and it's sort of accepted in the zeitgeist that at least some part of this um just kind of happens now and it requires substantially less of our creativity and input to

实现。

英文原文

achieve.

区分新建探索与既有维护

因此，这当然也就是我该呼吁大家请停止构建软件工厂的时候了，对吧？呃，所以我——我显然有点在开玩笑，但也同样是认真的。呃，我认为你们真的应该停止将自己的构想、雄心和目标强加于工厂流程之上。因为大多数时候我们并不清楚自己想要什么；或者说至少很多时候我们在创造新事物时，我们尚不知道自己想要什么。如果我们试图去把一个连自己想要什么都不知道的东西工业化，我们肯定无法得到正确的结果，或者会耗费太长时间，亦或投入过高成本。它会在多个层面令人感到遗憾。

英文原文

So that's of course the time for me to say please stop building software factories, right? Um, so, uh, so I and I and I'm I'm, you know, obviously, you know, a little bit tongue and cheek here, but, um, but also serious, uh, uh, I think you really should stop, uh, subjecting your ideas and your ambitions and your goals, uh, to a factory process. Um, you know, because most of the time we don't know what we want or at least a lot of the time we're creating something new, we don't yet know what we want. And if we're trying to industrialize something where we don't know what we want, um, uh, we're certainly we're not going to get the right thing or we're going to take too long or we're going to spend too much. Um, we're going to have all it's going to be sad in multiple ways. Okay.

所以我想说的是，让我们开始做一些甄别：我们究竟是在探索产品与市场契合度（PMF）、开拓全新领域、征服全新领地，还是在做一些已有先例且已经在正常运转的事情。对于我们正在以何种方式、何种模式工作保持这种辨别力，我认为是非常有帮助的。

英文原文

Um, so what I would say is, uh, let's begin to use some discernment about whether we're discovering product market fit, we're breaking new ground, we're conquering new territory, or whether we are doing something that, you know, already has a precedent and is already working. Okay. Um and that discernment about sort of what way, what mode we're working in, um uh you know, I I find it really helpful.

因为棕地（brownfield）工厂极好的一点在于，它是一项我们已经在获得报酬的业务。客户正用他们的真金白银和使用行为在表明：“是的，这对我有效。我喜欢它。我想要更多这样的东西。”这是那些维持正常运转、支付账单并保持客户满意的既有业务。而且你不需要去探索关于它的任何东西，因为闭环维持该系统稳定性所需要知道的一切，都已经内置在运行时中了。

英文原文

Um because, uh the great thing about a brownfield factory is it's something that we're already being paid for. It's something where customers are saying with their dollars and their and their utilization like, "Yeah, this works for me. I like it. I want more of this." Okay. Um it's the existing things that keeping the lights on, paying the bills, keeping keeping customers happy. Um, and you don't need to discover anything about it because everything you need to know to loop the stability of that system is is just built into the runtime.

所以正在发生的一切、客户使用它的所有方式、他们谈论它的一切、甚至可能包括工单支持，这些都在强化你在该棕地项目中已有成果的完整性、生命力以及效用。

英文原文

Okay, so everything that's happening, all the ways your customers are using it, everything they're saying about it, maybe even the support tickets, those are reinforcing um the integrity and the vitality and the utility of whatever you've got in that brownfield. Okay.

然后我们当然会讨论这个，但另一个令人兴奋的事情——正如我们从许多其他演讲者那里听到的那样，比如 Jared 刚才就详述了重写 Bun 和 Claude Code——你当然可以重写大量的棕地代码。这就涵盖了所有那些技术债务，所有那些在历史上拖慢我们步伐的累赘，以及所有那长到不可思议的待办列表（backlog）。我们现在能看到这些正在逐渐被清空。而且我们不必耗费太多精力去询问客户想要什么，因为客户每天都在到来，他们都在说：“我正在使用的就是我想要的。”

英文原文

And then of course we're going to talk about this but um uh the another exciting thing as we heard from many of our other speakers um uh you know Jared just uh talked all about rewriting uh bun um and and cloud code um you can rewrite a lot of that brownfield of course. Okay, so this becomes all of that technical debt, all those sort of boat anchors that are slowing us down historically. Um, uh, all that, you know, impossibly long backlog. Um, we can see those drying up now. Um, and we don't have to spend, uh, much energy asking what the customer wants because the customer is arriving every day and they're saying, "What I'm using is what I want."

所以，我们知道该构建什么。我们去构建既有的东西。

英文原文

Okay. So, um, so we know what to build. we build the we build the existing thing. Uh

将承诺编码为闭环

关于如何思考棕地项目的处理方式、清理它并使其具备自给自足能力的几点说明：在 Diffusion，我们使用“承诺”（promise）这个词，来代表我们在产品中所看重的某项保证、某个方面、某个优先级和某个维度。功能是一项承诺，东西的成本与定价是一项承诺，安全性也是一项承诺。而且你极少会想要违背或倒退自己的承诺。

英文原文

and a couple of uh a couple of notes about just ways of thinking about working with your brown field and you know cleaning it up and making it self-sustaining. Um uh at diffusion we use the word promise um to represent you know some guarantee some aspect some priority and some dimension within the product that we value. Okay. feature is a promise. Um, you know, the cost and price of a thing is a promise. Um, the security of a thing is a promise. Um, and it's rare that you want to revert and regress on your promises. Okay.

所以，如果你能够开始讨论我们对既有产品所抱有的各项预期，你就可以开始将它们编码进回路之中。无论你用 YAML 文件、某种 Python 代码还是图来表示，回路就像我们一直在讨论的任何工厂方案一样，它具备某种特征：接收输入，产生输出，然后再以某种方式反馈回去。正如你们许多人所知道的，这是放大器的理想化框图；如果切断那条反馈回路，你最终就只会放大噪声。我认为大家现在都有这种体会，尤其是我们一直在与软件工厂及反馈回路打交道。必须有某种方法将这种保真度反馈到系统的输入端。但如果你做到了并且调校得当，它就能良好运转。

英文原文

Um, uh, so if you can start talking about the ways that we have expectations that are against an existing product, um, you can begin to encode those in loops. Okay? And a loop just like any of the factory schemes we've been discussing whether you represented in a YAML file or some Python or graph um you know a loop has the characteristic of somehow uh taking an input and an output and then somehow feeding it back. Okay, as many of you will know this is the sort of idealized uh block diagram of an amplifier and if you cut that feedback leg you're just going to amplify noise. Okay. And um I think we all have this feeling now that we've been working with software factories and working with loops. Um there has to be some way of feeding back that fidelity into the input of the system. But if you do it and you tune it right,

它就能正常运作。所以这是我们应该努力追求的，尤其是在棕地环境中。我们正在获取修复、稳定和改进这些回路所需的一切信号。

英文原文

it can work right. Um so uh so this is what we should strive for, especially in a brown field. We're getting all the signal we need to cure, stabilize, and improve those loops.

持续运行与自我简化

棕地工厂的下一个属性是，它确实应当是持续不间断的。

英文原文

The next attribute of a brownfield factory is you know it really should be continuous.

如果你回顾过去的代码提交历史，那些提交存在某种季节性规律。会有每日和每小时的周期波动，也会有每周的周期循环，你会看到那些提交以固定的间隔进来。

英文原文

If you look at uh the history in the before times of when commits would arrive there was some seasonality to those commits. There would be the daily and hourly seasonality. There would be a weekly cycle and you would see those commits come in at intervals.

现在，当你的回路是由工厂里的智能体（agents）维护时，这一切可以持续不断地发生。我不知道，它可能看起来像一条直线。一长串全天候 24/7 都在发生的扎实活动，因为总有需要操心的事情。是不是有点慢了？是不是有点不安全了？是不是有点别的什么问题了？所有这些担忧都是全天候存在的。如果你以正确的方式构建你的棕地工厂，它应该与持续的智能应用、针对这一问题的持续求证相兼容。而且它应该就那么持续跑着。

英文原文

It can happen continuously now when your loops are being maintained by your agents in a factory. I I don't know like it might look like a straight line. Uh just uh just just a solid list of activity that's happening 247 because there's always something to worry about. Um is it a little bit slow? Is a little bit insecure? Is it a little bit whatever? Um all of those worries apply around the clock. And if you're building your brown brownfield factory the right way, um it should be it should be compatible with continuous intelligence, continuous application of this question. Um and and it should just run. Okay.

所以你有了现有的代码现状，有了一套既有的承诺。让它持续化，让它闭环运行，并让它坚守那些承诺。因此另一种概括方式是，我针对既有承诺集的棕地工厂，它应当就保持持续运转。它应当处于后台，并代我坚守那些承诺。

英文原文

So you've got an existing code of state, you've got an existing set of promises. Um just make it continuous, make it loop and make it keep those promises. Okay. So so another way to summarize that is you know my brownfield factory on an existing set of promises. It should just be continuous. It should be background and should be keeping those promises for me.

所以我强烈倡导构建这样一个系统。它将赋予你自由和创造力，让你开始去思考在绿地（greenfield）项目中应该做些什么。那可能不会带有同等程度的回路严密性，而可能会带着更多探索真正产品与市场契合度（PMF）的精力。

英文原文

Okay. So uh so I'm a strong advocate of building a system like this. Um, and it's going to give you the freedom and the creativity to begin asking what you should be doing in the green field. You know, probably not with the same level of looping rigor, probably with more true product market fit and discovery sort of energy. Okay.

好了。另一个非常迷人的特性。这当然是我在过去一年里切身体会到的。我之前提到过，一年前它能工作，但包含很多组件。让它运转起来在很大程度上依然属于工程维度的苦功。正如我们从 Zach 和 Potato 等其他人那里听到的，这些系统正变得越来越容易构建，这在很大程度上要归功于模型本身以及运行框架（harnesses）。但对于你的工厂，你可以期待的一点是，它会随着时间推移而不断简化。

英文原文

All right. One other really fascinating property. Um, this is certainly something that I've experienced uh over the last year. I mentioned earlier, you know, a year ago um it would work, but there was a lot of componentry. Okay. getting it getting it to work was still very much engineering. Um, as we heard from some others from Zach and from Potato, um, these systems are they're getting a little bit easier and a lot of credit goes to the model itself, um, as well as the harnesses. But one thing you can expect from your factory, um, is that it is going to simplify over time. Okay.

这是我们在自己的 skills 中见证过的事情。Skills 当然变得越来越短了。框架当然变得越来越通用了。这在很大程度上再次归功于模型本身。

英文原文

Um, this is something that we've seen in our skills. The skills are getting shorter, of course. The harnesses are getting more general, of course. Um, and a lot of this again is credit um, uh, to the models themselves.

我要提一下，有一家著名的人工智能公司也制造这些小构件。这是一个相当著名的演变历程。那家 AI 公司的 Raptor 1 发动机虽然非常繁复，但它确实能够工作。它长成这个样子。管道非常多，但它确实推进了火箭。在相同的封装尺寸内——大约相同的外部尺寸下——第二版性能更高，活动部件却更少。

英文原文

Um, I'll note that uh, there's a uh, there's a famous AI company um, that also makes these widgets. Um, and this is kind of a famous uh, progression. So the Raptor 1 engine from that AI company um, uh, was really fiddly but it worked. Okay. Uh, it looked like this. There were a lot of tubes but it definitely propelled the rocket. Okay. In that same package, right, that same approximate external dimension, the second version had higher performance and fewer moving parts.

从这一个版本到另一个版本的飞跃，是智慧的极大投入。就像真正的火箭科学家和工程师集中精力研究我们如何才能让这件东西更简单。这很难，召集这些人很难，让他们专注于一个棘手问题也很难。

英文原文

Okay, the delta between this one and this one was a tremendous application of intelligence. Okay, like literal rocket scientists and engineers focusing on how can we make this thing simpler. Okay, um you know that's hard. It's hard to get those people and it's hard to concentrate them on a difficult problem.

当然，我们在这种简化过程中拥有强大的盟友，因为我们有 Astra、Fable 和 Carbon。它们正在努力将我们的工厂从第一版推进到第二版、第三版，以便当我们回来，比如周一回到工作岗位之类的时候，我们的工厂可能比刚开始时要简单得多；或者我们只需再等待一个模型版本，我们的工厂活动部件就会更少，而性能却会提升一倍。所以在当前这个时刻，这是一个此前从未存在过的非常有趣的特性。因此，如果你能勉强让这套东西运转起来，你很快就可以期待获得更平滑的层流与更高的性能。无需你在其中投入太多创造性的贡献

英文原文

Um we have tremendous allies in this simplification of course because we have Astra and Fable and Carbon right and so they are trying to take our factory from version one to two to three so that by the time you know we come back we come back and you know start our jobs on Monday or whatever um maybe we have a much simpler factory than when we started right or we wait one more uh model version and our factory is has fewer moving parts yet twice the performance. Okay, so this is a really interesting feature at this moment that hasn't existed before. So if you can just barely get the thing to work, you can expect uh uh a smoother laminer flow, higher performance, you know, soon. Okay, without much of your creative contribution in

between。

英文原文

between.

好，那么呃，我们来建一个棕地工厂（brownfield factory）。给那座工厂的一个极好指令就是“照旧，但更快”。

英文原文

Okay, so uh let's make a brownfield factory. Um uh a great instruction to give that factory is the same but faster.

呃，我很少参加过有人去争论……呃，你知道的，去争论提升系统性能的会议，对吧？呃，只要你处于某种帕累托均衡（paro equilibrium）状态，没有用更快去换更便宜，也没有用更安全去换更快，好的，那这就是件好事。好的。所以，呃，所以它应该更快。它应该更便宜。它应该更安全。呃，请注意，有时我们会把“更好”也算在这个组合里。而在棕……在棕地工厂中，“更好”可能很难实现。“更好”可能需要融入更多你的意图。“更好”可能需要融入更多你的判断和品味。呃，所以也许那就是你要进入探索（discovery）并采用不同技术的地方。但天哪，更快、更便宜、更安全。呃，特别是在V启示录（V apocalypse）的另一侧。抱歉，另一侧。依然是在V启示录的开端。比如，我们、我们绝对需要这些东西自主地变得更安全，对吧？

英文原文

Um I've been in very few meetings where someone debated uh you know increasing the performance of a system, right? Um as long as you are at some sort of a paro equilibrium where you're not trading faster for cheaper or trading safer for faster, okay, then this is just good. Okay. So uh so it should be faster. It should be cheaper. It should be safer. Um, note that sometimes we include better in this mix. And better may be difficult to achieve in a brown in a brownfield factory. Better may involve more of your intent. Better may involve more of your judgment and taste. Um, so maybe that's where you go into discovery and use different techniques. But gosh, faster, cheaper, safer. Um, especially on the other side of the V apocalypse. Sorry, other side. Still at the beginning of the V apocalypse. Like we we definitely need these things to just autonomously get safer, right?

好的。所以你应该拥有那个。好的。所以，呃，所以我认为每个人都应该拥有棕地工厂。针对你现有的承诺，你应该拥有它。你应该拥有它。随着时间的推移，它应该变得更好、更快、更便宜。呃，我发现这大概需要一周时间。所以如果你还没花上那周时间，那就去做吧。就像无论你什么时候回到家，就像去、去花上一周时间，呃，就说你下周休假，然后在你所有现有的承诺上把你的棕地工厂立起来。呃，如果你没有办法做到这一点，或者你觉得需要帮助，你可以随时联系我，呃，我们随时准备提供帮助。

英文原文

Okay. So you should have that. Okay. So um so I think everyone should have the brownfield factory. You should have it for your existing commitments. You should have it. It should become better faster and cheaper over time. Um I find that this takes like a week. So if you haven't spent that week yet, just do it. Just like whenever you get home, like go go spend a week uh say you're on vacation next week and just stand up your brownfield factory on all your existing commitments. Um, if you don't have a way to do that or you feel like you need help, you can always reach out to me and uh we're standing by to help.

提升抱负：构建需求工厂

呃，所以棕地工厂，你们应该去做，但也太简单了。好吗？就像，拜托，太简单了。好吗？呃，呃，你们需要、你们也需要提升你们的抱负。好吗？所以在接下来的几分钟里，呃，我们将讨论如何提升我们的抱负。

英文原文

Um, so brownfield factories, you should do them, but also too easy. Okay? Like come on, too easy. Okay? Um, uh, you need you need to elevate your ambition as well. Okay? So, in my remaining couple minutes here, um, we're going to talk about how to elevate our ambition.

好的。所以，我认为我们所有人此时都在苦苦应对的一件事就是：天哪，产出了太多的代码。更糟糕的是，它们还能运行。呃，呃，东西太多了。呃，可能是太多东西了。呃，我们可能面临着东西过剩。而且，呃，情况可能是我们的客户，呃，并不想要所有这些软件。他们其实，他们其实并没有要求软件。他们要求的是更高的毛利率。他们要求的是更多的客户。他们要求的是更多的品牌差异化。好的。呃，而只是给他们交付更多供点击的窗口、呃，供点击的按钮，呃，那已经无法满足他们了。

英文原文

Okay. So, one of the things that I think is we are all struggling with at this moment is gosh, there's a lot of code coming out. Even worse, it's working. Um, uh, that's a lot of stuff. Uh, it's maybe too much stuff. Um, uh, we maybe have a glut of stuff. And, uh, it could be the case that our customers, uh, didn't want all this software. They actually, they actually weren't asking for software. They were asking for more gross margin. They were asking for more customers. They were asking for more brand differentiation. Okay. uh and just shipping them more windows uh to click on and buttons to click on. Um that's not doing it for them anymore.

好的。呃，而这就是我们所有人，呃，开始看到我们新工作浮现的地方。好的。我们可以非常顺畅地与模型沟通。这就是你们为什么会在这里，对吧？呃，你们公司里还有其他人，他们并不像你们这样熟练地与这些事物沟通。好的。呃，所以这意味着我们现在有了一份新工作。呃，我们必须想办法去建立一座需求工厂（demand factory），因为我们的棕地工厂无法构筑护城河，对吧？它所能做的就是保持我们现有的承诺良好运转。呃，这很棒，因为它让我们能够去关注大楼之外的事物，去面向市场、面向未来。呃，但是天哪，我们、我们需要开始着手建设需求工厂了。好的。呃，我并不真正清楚需求工厂是什么，但它、它很可能，呃，呃，看起来像这样。

英文原文

Okay. Um and this is where all of us um are beginning to see the emergence of our new job. Okay. We can communicate with the models quite well. That's why you're here, right? Um there are other people in your firms who are not as fluent in communicating with these things. Okay. Um so that means like we have a new job now. Um, we have to figure out how to build a demand factory because our brownfield factory can't build the moat, right? All it can do is keep our existing promises running well. Um, that's great because it allows us to focus on sort of outside the building and look toward the market and look toward the future. Um, but gosh, we we need to start working on the demand factory. Okay. Um, I don't really know what a demand factory is, but it it probably um uh looks something like this.

它看起来可能正是我们需要的东西，这样当我们在后端的软件工厂这里提升了性能时，我们才不至于把软件洒得满地都是。而且我们将需求系统与我们的生产系统进行速率匹配。好的，但是需求工厂中需要包含大量的探索。这是一个、这是一个新想法。好的，但我认为你们绝对就是那些能够开始推动你们的组织去提出这些问题的人。比如不要坐等销售和营销部门在这个问题上找上门来。比如你们可以看到，在你们品牌的范畴（envelope）内，你们还可以交付更多的东西。呃，但是你们需要去向外界解释这一点，并且你们需要用新的词汇、新的图像、新的产品线等等来解释它。

英文原文

It probably looks like the thing that we need so that when we boost performance in the back end here in the software factory, we don't end up spilling software all over the floor. And we rate match the demand systems to our production systems. Okay, but there's a lot of discovery that needs to go into the demand factory. This is a this is a new idea. Okay, but I think you are absolutely the ones who can begin pushing your organizations to ask these questions. Like don't wait for sales and marketing to come to you on this question. Like you can see that within the uh envelope of your brand, there's something more you can ship. Um but you're going to need to explain that to the world and you're going to need to explain it using new words and new imagery and new product lines and so forth.

我认为你们的需求工厂可以帮助你们向市场提出更好的问题，因为智能体（agents）可以协助排定接下来要问什么问题的优先级。我认为它可以帮助更好地倾听，因为智能体可以综合整理你们从这些问题中收回的答案，并能帮助你们重新排序，它们可以在夜间帮助你们重新排序，而且它们可以将其与你们听到的其他所有内容进行比较。好的。因此，我认为它们可以安排更好的实验。我们如何获得针对这一特定产品的需求价格弹性？你们拥有的实验带宽是有限的。你们如何去分配它？模型将在这方面帮助你们。我认为这可能会带来更低延迟的响应，使我们能够更快地给出肯定的答复。好的。并且客户开始不再自我设限，他们开始索求更多。那将是极好的。呃，我认为，很奇妙地，你知道，即便我们的竞争对手拥有相同的模型，也会存在某种新均衡的版本，在此之中我们所有人都在进行博弈论最优竞争，然而如果我们先起步，依然会有某种优势。好的，所以现在就开始行动，然后去摸索出我有可能向市场承诺的最大限度是什么。那就是我的品牌范畴，让我们去填满它。

英文原文

I think your demand factory helps you ask better questions of your market because the agents can help prioritize what to ask next. I think it can help with better listening because the agents can synthesize the answers you get back from those questions and can help you rerank and they can help you rerank overnight and they can compare it to everything else you've heard. Okay. And therefore I think they can schedule better experiments. How do we get the price elasticity of demand on this particular offering? You have a finite amount of experimental bandwidth. How do you allocate it? The models are going to help you with that. I think this could lead to lower latency responses where we are saying yes faster. Okay. and customers begin uncensoring themselves and they start asking for more. That would be great. Um I think like weirdly, you know, even though our competitors have uh the same models, there's some version of some new equilibrium where we're all doing game theory optimal comp competition and yet still if we start first, there'll be some edge. Okay, so get started now and then figure out what is the maximum extent of what I can possibly promise to the market. And that's my brand envelope and let's fill it up.

关于品牌范畴，稍微题外说几句。呃、呃，假设我们是SpinDrift。呃、呃，我们在卖一些带有一点果汁的很棒的气泡水。呃，我提议，嘿，你知道吗？咱们加点咖啡因进去吧。比如也许委员会批准了。比如也许我们把它交付上市了。呃，然后下周我走进来，我说：再加机油。好的。呃，你知道，那肯定会被否决。但是，呃，能量饮料，那是我们品牌范畴的一次良好扩展。对你们的每一家公司来说，情况都是一样的。也许你们卖不了机油，但也许你们可以卖能量饮料。所以去搞清楚你们的能量饮料是什么，现在就开卖吧。好的。所以去建起你们的需求工厂。呃，它将有大量的、高质量的信号从客户那里涌入，从市场上涌入，从你们可以衡量需求的任何地方涌入。好的。呃，它将拥有、它将拥有巨大的流量。好的。呃，但是为了让这种流量不至于压垮变革管理或意愿，呃，它不能撑爆你客户的需求。因此，你需要拥有更多的客户。好吗？事实上你需要拥有更多的一切，但你需要从这些客户身上获取更多的钱包份额（wallet）。好吗？为了在这些客户中获得更多的钱包份额，你将不得不销售更多的SKU。好吗？呃，除了更多的SKU之外，你还得将它们组织成条理清晰的产品线。所以，你将拥有更多的产品线。好吗？一旦你把所有这些都发挥到了极致，你就需要去直接收购你的竞争对手了。好吗？如果你现在就开始行动，你就会成为实施收购的那一方。

英文原文

And just quick digression on brand envelope. Uh um let's say we're SpinDrift. Uh uh we we're selling some awesome sparkling water with a little fruit juice. Um I propose, hey, you know what? Let's throw some caffeine in there. Like maybe that gets by the committee. Like maybe we ship that. Um and then the next week I come in and I say like and motor oil. Okay. Um you know, that's going to get shut down. But uh energy drink, that's a good expansion of our brand envelope. For every one of your companies, you have the same thing. Maybe you can't sell motor oil, but maybe you can sell an energy drink. So figure out what your energy drink is and start selling it now. Okay. So let's build your demand factory. Um it's going to have a high volume of high quality signal coming in from customers, coming in from the market, coming in from any place where you can measure demand. Okay. Um and it's going to be it's going to have a high volume of flow. Okay. Um but in order for that flow to not overwhelm change management, desire, um it can't blow out the needs of your customer. So therefore, you need to have more customers. Okay? You in fact need to have more of everything, but you need to have more wallet within those customers. Okay? To get more share of wallet within those customers, you're going to have to sell more SKUs. Okay? Uh in addition to more SKs, you're going to have to organize them into coherent product lines. So, you're going to have to have more product lines. Okay? And once you've maxed out all of that, you're going to need to just acquire your competitor. Okay? And if you get started now, you're going to be the one doing the acquiring.

好的。所以，呃，就是这样了。这就是棕地工厂。我认为你们可以建立起你们的棕地工厂。我认为从现在开始，你们绝对可以建立起你们的需求工厂。呃，我显然也认为你们可以重塑你们的产品管理，重塑并唤醒你们的高管团队。呃，我们在Diffusion随时准备提供帮助。所以，呃，我们确实在那里附带了一些我非常喜欢的关于棕地技术的技术资源。呃，并且，呃，是的，呃，我、我期待着与你们中的许多人建立联系。好的，保重。谢谢。

[掌声]

英文原文

Okay. So, uh, so this is it. This is brownfield factories. I think you can build your brownfield factory. I think you can absolutely build your demand factory beginning now. Uh, I also obviously think you can reboot your product management and reboot and you could wake up your executive team. Um, and we are standing by at Diffusion. So, uh, we do have some technical resources attached there on some brownfield techniques that I really like. Um and uh yeah uh I I look forward to uh connecting with many of you. Okay, take care. Thanks.

[applause]

TraceTune 框架介绍

Heat。

嘿，heat。嘿，heat。

Heat。 Heat。

Heat。

Heat。

一

英文原文

Heat.

Hey, heat. Hey, heat.

Heat. Heat.

Heat.

Heat.

one

大家好，欢迎来到我们的旧金山AI工程师（ag engineer San Francisco）演讲。我叫Steven Chin。和我在一起的是Cassandra Chin，她是T Trace Tune框架的创作者。

英文原文

Hello and welcome to our ag engineer San Francisco talk. My name is Steven Chin. I'm joined by Cassandra Chin, who's the creator of T Trace Tune framework.

嗨，我是Cassandra Chin。我构建Trace Tune是因为我想改进PR。我看到很多开发者正在做出很棒的事情。借助AI技术，创新的速度比以往任何时候都要快。但是很多拉取请求（poll requests）正遭到拒绝。这并不是因为代码不好。而是因为这些PR经常包含错误的信息，或者写得很糟糕。因此，希望借助Trace Tune，我们能够改善这一点。它还可以选择添加一个有趣的

英文原文

>> Hi, I'm Cassandra Chin. I built Trace Tune because I wanted to improve PRs. I see a lot of developers doing great things. The speed of innovation is as fast as it has ever been with AI technologies. But a lot of poll requests are getting rejected. And it's not because the code is bad. It's because the PRs often contain incorrect information or are written poorly. So hopefully with Trace Tune, we can improve that. It also has the option to add a fun

漫画，用风趣幽默的方式来帮助解释你的 Pull Request。

英文原文

comic to help explain your poll requests in a funny way. And

PR 质量与 Token 效率

我们今天要做的，是稍微梳理一下 PR 的常见问题，讨论如何提升质量、如何提高 Token 的使用效率，并希望能借此帮助每个人在各自的组织中成为更优秀的 AI 工程师。就像我们的主角 Alex 一样，他这辈子唯一想得到的，就是一件 AI 工程师连帽衫。

英文原文

>> what we're going to do today is we're going to go through a little bit about kind of the common issues of PR iss how to improve the quality, how to improve token efficiency on this, and hopefully help everyone to be a better AI engineer for your organization. Like our main character, Alex, who all he wants in life is to get an AI engineer hoodie.

那件连帽衫确实很帅。我有很多 AI 工程师的 T 恤，但还没有连帽衫。

英文原文

>> That's a really neat hoodie. I have a lot of AI engineer t-shirts, but I don't have the hoodie yet.

换过来做这件 AI 工程师连帽衫吧。大家都

英文原文

>> Get switched to make this AI engineer hoodie. Everyone

准备好了吗？嗯，这动力足够了，希望他能在大会上就他的案例研究做一场精彩演讲，讲讲他在公司为了提高 PR 质量所做的工作。这里是 Alex 工作的 Token 工厂的画面。Alex 和一群智能体一起工作。当然，他身后有财务部门在盯着，因为我们大家消耗的 Token 都太多了，他们想确保我们能控制住成本。嗯，他的搭档 Pip，走在他旁边的那位，带着所有的证据材料，负责查看归档记录和实际发生的情况。然后，嗯，Meera 负责所有的审查。所以，她一直在把控各项事务。

英文原文

>> ready? And um this is enough motivation that hopefully he's he's going to do an amazing talk at the conference about his use case study on what he's doing in his company to improve the quality of PRs. Here is a picture of the token factory where Alex works. So, um, Alex works together with a bunch of agents. He has, um, of course, finance looking over his shoulder because we we all spend too much tokens and they want to make sure we're keeping up with costs. Um, his buddy there, Pip, he's walking with carries all the evidence and looks into the the archives and what's actually happening. And then, u Meera does all the reviews. So, she's she's keeping track of things.

看来 Pip 把他的工位空出来了。

英文原文

>> It looks like Pip left his station empty.

工厂的一部分，直到……直到他看到 Alex 提交的 PR。噢，不。

英文原文

>> Part of the factory until until he sees what Alex produced for a PR. Oh, no.

那是一份非常长的账册。

英文原文

>> That's a very long ledger.

审查代码有太多的内容

英文原文

A lot to review

所以这很难。Meera 正在拿着细齿梳一点一点仔细过每一项。她上面确实标了红线，在 PR 里发现了一些问题。这是一个真实的 PR。我认为那实际上就是你的 trace tune PR 之一，它需要有针对 URL 的精确预览链接、命令归属、确保早期的工作也同样符合资格，并在 PR 记录中对所有这些进行详细说明，才能通过 Meera 非常挑剔的审查。

英文原文

>> the code. So, it's it's hard. And Merror's Merror's going through everything for the fine tuned comb. She actually has red lines, found some issues with the PR. And this is a a real PR. I think that's actually one of your trace tune PRs where it needs to have exact preview links for the URLs, command attribution, make sure the earlier work is um qualified as well and explain all this in detail in the PR record to be accepted by Mera's very critical review.

这很有意思，因为在构建 trace tune 的过程中，我实际上是一边构建，一边用 trace tune 来测试它自身。所以我让 trace tune 来生成这些 PR，而随着我对 PR 的改进，它也变得越来越好。

英文原文

>> This was interesting because as I was building trace tune, I was actually like as I was building it, I was using trace tune to test it. So I would have trace tune build the PRs and as I improved the PRs it would like get better.

一路上你还得到了非常棒的漫画，正好能在做东西的时候找点乐子。

英文原文

>> You got amazing comics along the way too just to have fun while you were building stuff.

[笑声]

英文原文

[laughter]

Token 消耗与上下文陷阱

所以，嗯，你可以看到，Alex 为了获得高质量的结果而不断重复提交 PR，但效果并不好。不过这倒是让他成了“Token 亿万富翁”。你了解“Token 亿万富翁”计划吗？

英文原文

>> So um you can see that the repeated PRs which Alex is doing to try to get good quality results are not working out well. It's working out for his token billionaire part. Do do you know anything about the token billionaire program?

了解，其实我自己就是个 Token 亿万富翁。为了构建 trace tune，我耗费了大量的 Token。

英文原文

>> Yeah, so actually I am a token billionaire. I spent lots and lots of tokens to build trace tune.

那可真有点不幸。那花的全是我的钱吗？

英文原文

>> That's kind of unfortunate. Was that was that all my money?

呃，别纠结细节了。

英文原文

>> Uh don't sweat the details.

好吧。所以，嗯，我认为在做 AI 工作时，把 Token 花在很多地方都是很值得的。不过，重复提交 PR 可能并不是你想消耗 Token 的地方，而且财务部门对此绝对不满意。所以他想要拿到结果，他想确保 PR 准确且有效，不用再通过查阅日志和反复重构证据来没完没了地重提 PR。

英文原文

>> Okay. So um I I think there's a lot of good uses for for tokens and for spending where AI work. um maybe doing repeated PRs is not where you want to spend your token and definitely the finance department's not happy with this. So he wants a result and he wants to make sure that the PRs are accurate are working and he has they don't have need to keep getting repeated again and again by reviewing the logs and reconstructing the evidence repeatedly.

这很有意思，因为通常情况下，当你只是让智能体写代码时，你处于自己的会话中，让它写一个 PR，它往往只是读取你的会话日志，利用其膨胀的上下文窗口试图弄明白自己原本应该做什么。实际上，随着会话轮次的推进，它经常会压缩自己的上下文窗口。所以随着过程的深入，它会变得越来越不准确。因此，我们在这里通过 trace tune 所做的，是让它去通读你电脑上的各个不同会话，真正弄清楚你构建了什么以及到底发生了什么。

英文原文

And it's interesting because normally when you're just like uh coding on your agent, you're in your session and you ask it to write a PR, it just reads your session log to try to figure out what it should have done using its inflated context window. It actually often like as it's turnurning it compresses its context window. So it gets more and more inaccurate as it goes along. So instead, what we're doing here is with trace tune, we're having it read through uh different sessions on your computer to try to really figure out what you built and what happened.

是的。我认为，像那种查阅审计轨迹、查阅智能体实际执行了什么的概念，和你之前为了教育目的而做的 show tail 项目有很多相通之处。

英文原文

>> Yeah. And I think like like that concept of um looking through the audit trails, looking through like what the agents have actually done is a lot of what you were doing with the show tale project you were doing for education.

没错。我最初其实就是从这个 showtail 项目开始的，因为当时我在支持一项学术研究。但他们遇到的基本问题是，学生们虽然写出了程序，却无法追溯他们实际付出的工作和心血，因为在 AI 编程中，你提交的只是最终成果，即程序本身；但大部分工作在于 Prompt 编写以及前面发生的一切，而这些并不会随着程序一起提交。于是我借鉴了其背后的技术和理论，心想：好吧，我也可以利用所有这些收集到的数据来改进 Pull Request。

英文原文

>> Yeah. I actually started this project with this showtail project because there was this like academic research study I was supporting. But basically the issue they hit is students were creating programs but there was no way to trace back the actual work and effort they did cuz with AI coding the what you submit is the final work the program but most of the work is the prompting and everything that came before and that doesn't get submitted with the program and I use the technology and theory behind that to be like well I can also improve poll requests with all this data collection.

太酷了。我认为 Pull Request 方面的改进是一件让所有人受益的事情，因为现在你可以找到证据，比如在执行 cd 之前记录当前工作目录，解释为什么其他测试数据可能会泄露到其他代码库中，并解释 PR 背后发生的事情；而不是像大多数 AI 生成的 PR 那样，仅仅是对 diff 进行逐行解释，毕竟它们就是基于 diff 构建 PR 的。但错误的事实仍会污染这些信息，可能导致冲突 PR 带来的问题，并最终依然给你返回不准确的结果。所以也许我们对此有解决方案。你怎么想？你有什么解决方案吗，Cassandra？

英文原文

>> Cool. And I think that improvement in pull requests is something that benefits everybody because now you can find the evidence like um logging the change directory before the or the CWD before the CD runs. explaining why maybe other test data is leaking into other repos and explaining what's happening behind the scenes on the PR rather than what most AIPRs read like which is a line by line explanation of the diffs since that's what they build the PRs off of but incorrect facts can still pollute the information um can cause a problem with conflicting PRs and then still give you back inaccurate results. So maybe we have a solution for this. What What do you What do you think? Do you have a solution, Cassandra?

有啊，大家都应该用 trace tone，因为里面有漫画。

英文原文

>> Yeah, everyone should use trace tone because of the comics.

所以，[笑声] 是因为漫画，而不是因为它能提供更好的 PR，对吧？

英文原文

>> So [laughter] because of the comics, not because it gives you better PRs, right?

呃，我是从漫画开始做起的，然后像是倒推着想：噢，我实际上需要把 PR 做得更好。

英文原文

>> Well, I started with the comics and I kind of went backwards and like, oh, I need to actually make the PRs better.

好吧，希望到最后既有更好的 PR，有更好的漫画，或许还能更好地节省 Token。也许

英文原文

Well, hopefully better PRs, better comics, and maybe better token savings if by the end of this. Maybe

本该如此。

英文原文

>> it should be.

Trace Tune 的工作流与局限

所以，嗯，trade students 是一个开源项目。它允许你通过引入智能体上下文、引入代码的相关信息，然后由一个独立的编写者和独立的审查者来构建更高质量的 PR，从而提升 PR 的质量；此外还可以选择附带生成精彩的漫画——Cassandra，你觉得这里最好的特性是什么？

英文原文

>> So, um, trade students an open source project. It lets you improve the quality of your PRs by pulling in agent context by pulling in information about the the code and then having an independent writer independent reviewer build better quality PRs plus optionally giving you an amazing comic which what what do you think the best feature is here Cassandra?

我觉得显然是漫画。这是我当初用 Traune 构建它时的早期漫画之一。

英文原文

>> I think it's obviously the comics. This is one of the earlier comics from when I was using Traune to build it.

真不错。而且你还确保了每集都有新角色，绝不出现重复的角色。

英文原文

That's good. And then you you made sure that there's new characters every episode. None of this repeating characters.

是的，那个 PR 就是关于这个的。

英文原文

>> Yeah, that's what this PR was about.

好的。接下来这是对最初 trace tune 流程的描述。它会找到特定分支的所有 trace。它对它们进行研究，审查描述，添加更多上下文，像这样循环进行，直到获取生成高质量 PR 所需的全部证据。嗯，无论是主观感受上，还是通过 PR bench 这个测试框架来看，它都能在 PR 中提供更好的质量、更深入的研究，以及更低的差错率。

英文原文

>> Okay. And then this is a description of the original trace tune flow. So it finds all the traces for a particular branch. It researches them, reviews the description, adds more context, and kind of goes in a loop until it gets all of the evidence it needs to make a a quality PR. And um subjectively, but also through um PR bench, which is testing framework, um gives you better quality, better research, lower um rate of inaccuracies, and the PRs.

我觉得在这一点上，生成的 PR 已经比不用 trace tune 要更好了。但我发现它偶尔还是会犯一些错误。你可以在最后看到，它试图加入更多上下文来进行修正。我觉得问题往往就出在那里。

英文原文

>> I think at this point the PRs were already better than if you don't use trace tune. But I found that it still makes some mistakes occasionally. You can see at the end that it tries to revise with more context. I think that's where the mistakes come in.

好的。嗯，Alex 觉得这样就足够好了。他打算尝试用 trace tune 来解决工厂的 Token 危机，看看这能否解决他所有的问题。打磨完毕，检查通过。Alex 看起来非常、非常开心，

英文原文

Okay. Well, Alex, Alex thinks it's good enough. He's going to try to solve the token crisis the factory with trace tune and see if this will solve all of his problems. Polish checked. Alex looks very, very happy,

但是 Meera 看起来没那么高兴。就像我刚才说的那样，看来我们在这里依然碰到了错误。原来最初的 PR 声称之后没有记录到运行通过的情况。这里的症结在于，实际的 trace 显示它在运行 bun test 的同时还在运行 bun xtsc。而这两个命令是不能同时执行的。它们必须按先后顺序执行，测试才能通过。

英文原文

but Meera doesn't look that happy. It looks like we still hit a mistake here like I was saying earlier. So the original PR claim was no later passing run is recorded. And the problem here is the in the actual trace it shows it running a bun test at the same time as the bun xtsc. And you can't run those two at the same time. They have to be run sequentially for the test to pass.

是的。确实如此。好吧，所以他声称测试通过了，并且运行了所有检查，但实际上代码根本不是这么执行的。

英文原文

>> Yeah. Exactly. Okay, so he claims test passing and all the checks were run and actually that's not what the code did.

可怜的 Alex。又一个被驳回的 PR，还有一只因为 Token 成本而极度愤怒的獾。嗯，这感觉有点像周一早上 Neo4j 的财务部门每周查看我的 Token 开销时的情景，不过他们非常宽容，尤其是对于像这样重要的项目。所以，也许 Alex 可以通过使用更便宜的编写模型来节省一些成本——这是财务部门最常要求我做的事情。他们总是说：“嘿，别再用 Opus 了。用 Haiku 吧，请换成更便宜的模型。这样行得通吗？”

英文原文

So poor Alex. Another failed PR, a very angry badger with all the token cost. Um kind of kind of makes me feel like Monday morning with the Neo Forj finance department looking at my token spend each week, but they're they're very forgiving, especially when it's a important project like like this. So maybe Alex can save some cost by using cheaper writers is the number one thing finance asks me to do. They're like, "Hey, hey, no more opus on it. Haiku, do something cheaper, please. Does this work?"

我感觉如果像那样做的话，质量会有所下降。仅仅削减成本并不能

英文原文

>> I feel like the quality is a bit lower when you do it like that. Just reducing the cost doesn't

帮上忙。

英文原文

quite help.

是的。呃，所以我们本来是想获得更好的质量，结果仅仅通过使用更便宜的编写模型，反而得到了更差的质量。

英文原文

>> Yeah. Uh, so we were trying to get better quality and we we ended up getting worse quality by just using cheaper writers.

显而易见，Alex 现在陷入了困境，对吧？他在修改上花得太多了。他没有降低 token 成本的解决方案。他在 PR 上没有拿到想要的结果。不过，也许 Pip 有什么能帮上我们的。Pip、Pip 有个朋友。

英文原文

And obviously Alex is in a bind here, right? He's he's spent too much on revisions. He doesn't have a solution for reducing token costs. He's not getting the results he wants out of his PRs. But maybe there's a there's something Pip can help us with. Pip Pip has a friend.

我很好奇这位朋友会是谁。

英文原文

>> I wonder who this friend could be.

噢，嗯，我想、我想从我之前的演讲里你们就知道，这位朋友肯定会是一个图。

英文原文

Oh, well, I I think I think you know from my previous talks that that this friend is definitely going to be a graph.

你就是那个搞图的人。我们又要搞另一个图了。

英文原文

>> You're the graph guy. We're doing another graph again.

嵌入式图数据库 Archie

是的。是的。总会有一个新的图。但这并不是同一个老图。这不是 Neo Forj 数据库。这是我们的新朋友 Archie，它是一个用 Rust 编写的嵌入式数据库，名叫 relate，实际上这周正在 NODES 大会以及旧金山 AI Engineer 的这场演讲上发布。它是一个开源数据库，能让我们在无需采用重量级方案的情况下，获得更高准确率、多跳查询以及更具解释性的跟踪记录等诸多好处。所以它有点像是一个专门针对这类嵌入式使用场景、面向 Agent 的图数据库。

英文原文

>> Yeah. Yeah. There's always a new graph. But this is not the same old graph. This is not the the Neo Forj database. This is our new friend Archie who is a embedded database written in Rust called relate that actually is being launched this week at the nodes conference and also this talk at a engineer in San Francisco and it's an open-source um database which allows us to get a lot of the benefits of more accuracy multihop queries more explainable traces is without having the heavyweight approach. So it's kind of an agent focused graph database specifically for these sort of embedded use cases.

好，那我们来看看 Archie 能怎么帮我们。之前我们很大程度上依赖 Agent 以及 Agent 循环来解决问题。但图数据库可以存储关系，可以找出聚类在一起的不同事物，构建一份将所有这些信息连接在一起的摘要，然后为 Agent——哪怕是更便宜的 Agent——提供更好的信息和更多的上下文，从而做出良好的起草决策。Archie 的好处在于他完全没有 LLM 成本。全都是本地计算和存储成本，而且它是一个完全内存式的数据库，用 Rust 编写，所以高度优化。因此，与我们之前承担的所有 LLM 和 token 成本相比，我们基本上是免费获得了关于所有这些提交更改的出处信息以及这个证据图。这意味着我们的伙伴 Alex 不需要再去调研了。他不需要再去梳理并重新找出所有事实、查遍所有来源，因为 relate 会替他完成所有这些繁重的工作，并构建一个可用于 TraceTune 的证据包。

英文原文

Okay, so let's see how Archie can help us out. So before we were relying a lot on on agents and loops of agents to resolve the problem. But graph databases can store relationships can find like like different things which are clustered build a brief which connects all this information together and then give the agent even a cheaper agent better information and more context to give a good drafting decision. And the good thing about Archie is he has zero LLM costs. It's all local compute cost and storage and it's a fully immemory database and rest so it's highly optimized. So we get this providence about all these submitted changes and this evidence graph basically for free compared to all the LLM and token costs which we were doing before. And this means our our buddy Alex doesn't need to research. She doesn't need to go through and find all the facts again and go through the sources because relate is going to do all that hard work for him and build an evidence packet that he can use for trace tune.

听起来真的很棒。也许我们确实需要更多的图。

英文原文

>> So, all sounds really great. Maybe we do need more graphs.

听你这么说我挺惊讶的，毕竟你刚听了我那么多场关于图的演讲。好，不过我们可以通过邻域摘要来做到这一点，并让它变得更好。也就是把所有的“为什么”、代码和检查项拿过来，在它们之间建立归属和关系，然后将一组相互关联的信息作为数据包返回。这样一来，当 Agent 起草 PR 时，它在一个单一来源中就汇集了所有关联事实及其出处，这使得编写模型起草时以更少的轮次达到更高的准确度，同时也让审查模型的工作轻松得多。所以我们不需要昂贵的审查模型。我们可以通过使用几个 Haiku 审查模型来节省开支，而不是用 Sonnet 审查模型，或者在 Codex 的情况下用 Luna 来得出审查发现、拿它们跟草稿比对、重新编写；然后我们保留更高质量的编写模型——Sonnet 或同级别的编写模型——以确保我们有一份能通过人工批准的高质量摘要。

英文原文

>> That's I'm surprised to hear that from you after all the graph talks you about to put up with. Okay, but we can do this and make it even better with the neighborhood brief. So basically taking all of the the Y the code and the checks putting attributions and relationships between those and then returning back a connected set of information as a packet and by doing this then when we're the agent is drafting up the PR it has all the connected facts with their sources together in a single source and that makes the writer draft more accurate with fewer passes and it also makes the reviewer's job much easier. So we don't need an expensive reviewer. We can save by using a few haiku reviewers instead of a um sonnet reviewer or or a luna in the case of codeex to find come up with the findings check those against the draft redo the writing and then we retain the higher quality writer a a sonnet or terror class writer to make sure we have a high quality brief that will pass human approval.

我们在构建这个的时候，让编写模型的质量高于审查模型感觉非常反直觉。

英文原文

When we were building this, it felt very counterintuitive to have the writer be high higher quality than the reviewers.

实际上大家普遍总是希望规划者、审查者、核对者是最高质量的模型。但如果你要做高质量的写作，一个较低级别的模型无论经历多少轮迭代、获得多少反馈，都是做不到的。它只会把事情变成清单，让内容显得非常机械。所以要写出有人类感的内容，你确实需要一个更高质量的模型。而审查和交叉比对步骤实际上更加机械化。所以这有点反直觉，但效果相当好。同样有点反直觉的是，呃，我们的獾胸口竖起了一个大拇指，不过我们还是忽略这个高质量的 AI 画作吧。

英文原文

>> It actually it's common that people always want to have the planner, the reviewer, the checker to be the highest quality model. But if you're doing high quality writing, a lower model, no matter how many iterations and how much feedback you give it, can't do it. It just turns in things into a checklist and makes it very mechanical. So to give writing which feels humanlike, you do need a higher quality model. And it's it's actually the the reviewing and the cross-checking step which is more mechanical. So it's a little counterintuitive, but that works out quite well. It's also a little counterintuitive that um our badger has a thumbs up coming out of his chest, but let's let's ignore the high quality AI artwork.

好，所以我们已经根据证据核对了所有的断言。我们用较便宜的审查模型查找遗漏的发现。完成了聚焦的工作，然后找到了互相关联的信息，为 Merror 提供更好的起草和审查。

英文原文

Okay, so we've checked all the claims against the evidence. We looked for missing findings with our cheaper reviewer. Did the focus work and then found the things which were connected information to give a better draft and review for mirror.

这就是输出结果。它包含了“为什么”。也就是日志中的 CWD，它描述了 CD 运行前的 shell 状态。异常是什么？也就是 CD 目标在仓库内，或者命令提到了路由。什么通过了？包含类型检查，加上在提交的内容上而非之前检查的 71 项测试。然后它从 correlate.ts 开始，检查开头的 CD 异常。Merror 看起来对这些结果相当满意，对吧？

英文原文

And here's the output. So it has the the Y. So log CWD and that describes the shell before the CD runs. What's the exception? So the CD targets in repo or the command mentions a route. What passed? It has type check plus 71 test checked on submitted content not before. And then it starts in correlate.ts inspecting the leading CD exception. And Merror looks pretty happy with these results, right?

是的，这可能是她真正感到开心的第一个场景。

英文原文

>> Yeah, this might be the first scene where she's genuinely happy.

PRBench 基准测试与成本优化

我想她在意识到 Alex 其实是个笨蛋之前，对那件工程师运动衫还是挺满意的。好，这里是使用 PRBench 进行横向对比分析的一些实际数据和结果。PRBench 是一个拉取请求基准测试，它结合审计跟踪记录、代码以及大语言模型评判来审查工作质量。当然，同样由人工复核准确性。你可以看到，在 Claude 和 Codex 上，在 1 到 100 分的评分体系下，你都能获得显着的质量提升。评判们总体上几乎一边倒地更青睐 TraceTune 加上 relate 的质量。嗯，在 1 到 100 分的评分体系上，大约是从 74 分到 76 分、82 分到 86 分。也就是提升了 2 分到 4 分。

英文原文

>> I think she was kind of happy with the the engineer sweatshirt before she realized that Alex actually was a was a goof. Okay, so here are some of the actual data and results from the sideby-side analysis using PRBench, which is a um pull request benchmark, which uses audit traces plus code and um LM judges to review the quality of the work. Of course, reviewed by a human as well for accuracy. And you can see that on both claude and codeex, you get a significant quality improvements on a scale of 1 to 100. The judges by and large almost unilaterally liked the trace tune plus relate quality better. Um on a one to 100 scale it was like 74 to 76 82 to 86. So plus two plus 4 point.

这些测试给我的 token 亿万富翁卡出了不少力，因为我们跑了大量的测试。我们专门为了测试创建了这个叫 PRBench 的附带项目，它会让不同的 Agent 在不同的拉取请求上进行测试并对比结果。起初我让它是串行运行的。做完一个测试，结束之后再做下一个，但后来我们决定把它们全并行跑起来。这是一种迅速达到高支出的方式。

英文原文

>> These tests contributed to my token billionaire card because we were running lots and lots of tests. We created this side project called PRBench just for testing and it we would have different agents test over different poll requests and compare the results. Initially I had it like running like sequentially. It'll do one test and finish it then do the next but then we decided to run them all in parallel. It's a very quick way to reach high spend.

是的。挺逗的。按照目前的测试速率，加上用例数量、测试数量和基准测试，呃，我们能用 Linux 所能同时处理的极限线程数，完全占满我那台高端游戏电脑，当然 token 支出也比我们预期的略高了一点点。但这一切都是为了改善全世界拉取请求的质量。所以你们都能从所有这些测试运行以及最高质量的改进中受益，而且你们能省下 token。我们已经替你们预先消耗了这些 token，因为采用图方案的 TraceTune 所消耗的 token 数量大约只有之前的一半，这是图带来的绝妙成果，因为现在你是在预先计算，提前获得了构建好的所有这些证据图。编写更便宜，审查也更便宜，而且结果质量更高。

英文原文

>> Yeah. So that's kind of funny. I I at at at at at the current testing rate with the number of use cases and tests and the benchmark um we can fully saturate my my high-end gaming computer with as many threads as Linux can handle simultaneously and and of course the token spend is is slight slightly higher than what we'd like. But it's all for the good of improving the quality of pull requests for the world. So you all get the benefit of all these test runs and having the highest quality improvements and you save on tokens. So we've spent the tokens for you in advance because trace tune with the graph solution takes about half as many tokens as it did previously and which is an amazing result from the graph because now you're premputing you're getting all this evidence graph construct in advance. The writing's cheaper, the reviewing's cheaper with a higher quality result.

我起初对这整套图的东西并不确定，但在看到成本几乎腰斩之后，这就毫无疑问了，它让这个程序运转了起来。

英文原文

>> I wasn't sure about the whole graph thing initially, but after seeing the cost tank almost in half, like it's not even a question, it's making this program work.

这是我们为 TraceTune 项目做的事情，但只要你有数据、有关系、有证据、有想要对比的事物，你几乎可以在任何项目上这么做，因为 Agent 并不那么擅长做数据分析、做所有的评估和整理。所以如果你在前端帮它们一把，使用像 relate 这样的内存数据库创建一个定制的图层，突然之间你的成本就会下降，你的质量就会上升，而你也能像我们的朋友 Alex 一样成为 token 亿万富翁。好吧，我想现在他把它当书签用了。我觉得这是个不错的用法。这是一张质感非常高级的书签。比信用卡还要重。

英文原文

>> And this is something we did for the trade student project, but you can do this for pretty much any project where you have data, you have relationships, you have evidence, you have things which you're trying to to compare because agents are not that good at doing the data analysis at doing all the evaluation and the curation. So if you give them a help at the front end by creating a a custom graph layer with an inmemory database like relate suddenly your costs go down your quality goes up and you too can be a token billionaire like our friend Alex. Well I guess it's a now he's using it as a bookmark. I think that's a good use for it. It's a really high quality bookmark. It's heavier than a credit card.

确实是。在 PRBench 上花了那么多 token，这确实是个不错的惊喜。所以，财务部门满意了。更低的起草成本，他们能合理提供资金。Alex 满意了。他有了新书签。他现在有了一个极佳的案例研究，通过关联进行事实的邻域检索，从而形成更好的摘要，聚焦审查，让审查模型获得同样精炼的摘要，让 Merror 确切获得她需要的证据，然后这就是 Alex 站在某个地方的 AI Engineer 舞台上，更好的 PR，更低的起草

英文原文

>> It is. It is a nice surprise for um spending all those tokens on PR Bench. So, finance is happy. Lower drafting costs that they can reasonably fund. Alex is happy. He has his new bookmark. He is now has a great case study with the connections doing neighborhood retrievable of facts so that it's a better brief focusing the review so the reviewers get the same compact brief ma gets exactly the evidence she needs and then here's Alex on stage at um AI engineer somewhere in the world better PRs lower drafting

结合图数据库降低成本，再加上开源构建跟踪（trace to builds），这还能为你生成精彩的漫画。

英文原文

cost with a graph plus open-source trace to builds, which also gives you amazing comics.

好了，让我们把热烈的掌声送给我们的朋友 Alex，他现在赢得了一件 AI 工程师连帽衫。

英文原文

Okay, so big round of applause for our friend Alex, who's now earned the AI engineer hoodie.

我觉得这绝对是他在服装上的一次升级。

英文原文

>> I think that's definitely a wardrobe upgrade for him.

是啊，我觉得他之前那种口袋保护套（pocket protector）的书呆子造型并不怎么适合他。所以，真替他高兴。我想我们很快就会看到 Swixs 穿着其中一件连帽衫登台了。他没法不穿了，毕竟我们都在他自己办的会上挤兑他去穿了。十亿 Token 计划非常棒。非常感谢团队想出这些方法来激励和奖励那些处于 AI 使用前沿、并摸索着如何让我们其他人用得真正高效的人。而且，呃……

英文原文

>> Yeah, I don't think the pocket protector look was that great for him. So, good for him. I think we'll see Swixs on stage with one of these hoodies soon. He he can't he can't not do it now that we shamed him into doing it at his own conference. The token billion program is amazing. A big kudos to the team for coming up with ways to incentivize and reward people who are on the bleeding edge of AI usage and learning how to make it really effective for the rest of us. And um

现在你也可以拥有更棒的 PR 了。通过提交调研充分、编写高效且成本更低的 PR 来给你的同事留下深刻印象——

英文原文

now you too can have better PRs. Impress your co-workers by having wellressearched effectively written and lowerc cost PRs

还带漫画

英文原文

>> with a comic

还带漫画。想了解更多信息，请访问 tracetune.com，了解一下 TraceTune。去试试看吧。它可以与 Claude 配合使用，可以与 Codex 配合使用，也可以与你所有喜欢的 AI 工具配合使用。如果你想在自己的项目中取得同样惊人的成果，试试 Relate 吧，这是我们全新的基于 Rust 的嵌入式开源图数据库，它能为你的智能体带来上下文，并帮助你构建出开箱即用、更加高效、成本更低且效果更好的应用程序。非常感谢大家来参加我们在旧金山举办的 AI 工程师演讲。如果你本周正在旧金山观看这场演讲，周六你该带孩子们去哪儿玩呢？Cassandra，

英文原文

>> with a comic. So for more information go to trace tune.com learn about trace tune. Give it a try. It works with quad. It works with codeex. It works with all your favorite AI tools. And if you want to do these same sort of amazing results on your own project, try out relate, our new rustbased embedded open-source graph database, which brings context to your agents and helps you build more efficient applications that just work better, are cheaper, and give you better results. So, thank you all for joining our AI engineer San Francisco talk. And if you're watching this talk this week at San Francisco, what should you do this weekend with your kids on Saturday? Cassandra,

我们正在举办一个 AI 工程师儿童日（AI engineer kids day）。所以，任何有空的人，直接给你们的孩子报名就行。我们要开办儿童研讨工作坊。孩子们将学习如何进行 AI 编程来制作他们自己的游戏。我想我们还要做一个……不，只有做游戏的那个。不对，我们也会做机器人的。

英文原文

>> we are running an AI engineer kids day. So, anyone who's available, just sign your kids up. We're teaching kids workshops. Kids will be learning how to do AI programming to make their own games. And I think we're also doing a No, it's just the gaming one. No, we'll do we'll do the robots, too.

好。是的，我们也会做机器人。孩子们将能使用 AI 来控制这些小易拉罐机器人。

英文原文

>> Okay. Yeah, we'll also be doing robots. So, kids will get to control these little soda can robots using AI.

是的。所以，满满的乐趣，具身 AI、游戏 AI 全部集中在一处。呃，可以在 AI 工程师儿童日报名，而且，你知道的，带上你的孩子，带上你的侄女、侄子，让我们一起来培养下一代 AI 工程师吧。

英文原文

>> Yeah. So, a lot of fun physical AI gaming AI all in one place. Um, sign up at AI engineer kids day and um, you know, bring your kids, bring your nieces, bring your nephews, and let's train the next generation of AI engineers.

好的。非常感谢大家，祝大家参会愉快。

英文原文

All right. Thanks so much and enjoy the rest of the conference.

利用 KV Cache 打造智能体安全防线

大家好，我是 Pinter AI 的创始人 Arius。呃，这场演讲是面向那些正在构建智能体并且运行着自己的推理技术栈，或者正打算这么做的人。如果你正是如此，你通常会将 KV Cache——现代推理引擎的立身之本——仅仅看作一种吞吐量优化手段。每个 Token 计算得到的 Key 和 Value 被保留下来，这样模型就不用重复计算它们了。今天，我想从一个不同的视角来展示 KV Cache。

英文原文

>> Hi, I'm Arius, founder of Pinter AI. Um, this talk is for people who build agents and run their own inference stack or plan to. And if that's you, you associate typically the KB cache, the bread and butter of modern inference engines usually as a throughput optimization. The keys and values computed per token kept so the model doesn't compute them again. Today I like to show KD cache in a different light.

AI 智能体应用出状况的方式越来越多，对吧？提示词注入、越狱、破损的工具调用、敏感工具调用、传感器数据泄露、隐藏后门以及漂移等等，对吧？所以目前我们防御这些可能导致 AI 智能体应用出错的诸多途径时，通常的做法是在边缘进行防护，比如通过引入一个裁判模型（judge model），根据你的策略来检查提示词输入和响应输出。对吧？或者你针对每种情况分别构建一个分类器。但这里提供了一个不同的视角：模型在行动之前、在解码之前、在生成响应之前、在该响应变成另一次工具调用之前，都会将每一个输入转化为计算。KV Cache 就是你的引擎对那次计算的记录，而现代推理引擎免费地使用它并读取它。

英文原文

The way agents agent AI agent application could go wrong are keep growing, right? Prom injection, jailbreaks, broken tool calls, sensitive tool calls, sensor data exposures, hidden back doors and drips and so forth, right? So the way right now we typically guard against against these numerous ways that an AI agent application could go wrong is by guarding the edge by perhaps passing a judge model that checks the prompts the inputs and the response outputs against your policy. All right? Or you build a classifier for each case. But here is a different light. The model turns every input into a computation before it acts before it decodes before it generates the response before that response becomes another tool call. The KV cache is your engine's record of that computation that modern inference engine for free are using it and reading it.

因此在本次演讲中，我想建议的是，作为对你们目前使用自身安全护栏（guardrails）方式的补充，利用这些数据并将其与你们的 KV Cache 基础设施集成起来。我把这称为“KV 优先的安全信号”（KV-first security signals），核心在于有三个重要的理由，对吧。首先，数据早就存在了。你是从推理技术栈原本随手扔掉的废料堆里拾取珍珠。其次，它覆盖了安全护栏的盲区，因为无论什么东西溜过了那些护栏，它依然会转化为计算并留存下来。因此，它非常适用于数据隐私应用，适用于你没有权限读取输入和输出的场景。实现方法相当直截了当，对吧？它分为三个步骤：你为你的人工智能工作负载、为你的 AI 推理流量学习一个基线，了解你的 KV Page 通常是什么样子；然后找出相比于基线属于低频罕见的 KV Page，这通常就是你的异常；接着你甚至可以对它们进行分类和解释。那么首先，让我们从该集成可以在技术栈的什么位置切入开始讲起。

英文原文

And therefore in this talk I would like to suggest as as a complement to the way you're already using your own guardrails to use this data and integrated with your KB cache infrastructure. I refer to this as a KB first security signals and basically three reasons that matters right. So the data is already there. You pick the gems from the pile your inference stack already was throwing away. It covers the guardrails blind spots because whatever slips past those guardrails, it still becomes computation and it stays in place. So it's perfect for data privacy application for cases where you do not have the luxury to read the inputs and outputs. And the recipe is pretty straightforward, right? It has three steps. You learn a baseline for your AI workload, for your AI inference traffic, how your KV page typically look like and then find the KV pages that are rare against your baseline typically your anomalies and then you can even classify and explain those. So first let's start with where in this stack this integration could happen.

关于 KV Cache，有趣的一点在于，它们的存在在某种程度上让推理变成了有状态的（stateful），对吧。例如，像 LMCache 这样的 KV Cache 管理层会将它们从 GPU 转存到远程存储；或者像 LLM-D 和 Nvidia Dynamo 会将每个新请求路由到可能已经持有该缓存 KV Page 的 Worker 上。所以有两个集成点，你可以把这些异常检测或那些分类器集成进去：要么你可以直接在你的推理引擎（vLLM 等等）内部实现它。这种方式的妙处在于它是实时的，而且它可能生成的警报可以伴随你未来的工具调用或未来的策略一同传递。这样一来，仅凭在推理引擎内部观察到的计算异常，你基本上就能强化你的策略算法。

英文原文

The interesting thing about the KB cache is that in a way existence of those had made the inference in a way stateful right. So for example things like KB cache management layers such as LM cache stores them from GPU down to remote storage or LLMD and Nvidia dynamo route each new request towards the worker that already holds perhaps that cached KV page. So there are two integration points you you you can integrate those anomaly detection or those classifiers into either you can implement it directly inside of your inference engine vg line and so forth. The beauty of this approach is that it's real time and and the the the alert that it would perhaps generate could ride along your future tool calls or your future policies. So you can harden basically your polic algorith but just generating the computational anomalies that you observe within your inference engines

或者你也许可以把它作为一种威胁发现的手段进行集成，因此你可以将其与你的 LMCache 集成，让它读取存储的 KV Page，并找出与其余部分不相称的内容，对吧。那么现在我想给你们展示一些数字，以及一个非常简单的 Hello World 用例，看看这是否真的有效，这是否是一种可行的方法。为此，让我采用一个完全没有文本可供过滤的用例：语音智能体。

英文原文

and perhaps you can integrate it with as a as a way of thread discovery in a sense so you can integrate with your LM cache so it reads the stored KB pages and find things that don't belong with the rest right so now I want to show you some numbers and and just a very simple hello world use case if this is actually working this is a valid approach and for that let me use with a use case that has no text to filter at all a voice agent.

这里就是一个语音智能体的案例研究。在这个案例中，使用的是 Gemma 4 E4B。我们的基线是 72 段常规音频剪辑，由四种声音讲出。我们观察的是 16-token 的 Page。显然这种粒度是可以改变的，但在这里我们只是使用了较短的 Page，以便基线中能有更多的数据。然后在测试中，我们使用了不同的声音，其中一个是纯净的声音，而另一个声音内部带有一条隐藏指令。正如这里所展示的，你不仅可以通过简单的异常检测来发现这些情况，而且你还能知道具体是哪一部分——呃，哪个 Page，你的请求的哪一部分发生了这个异常事件。我们甚至更进一步，训练了一个将 KV Page 作为特征的模型，并且我们没打算解释它或用有用的内容给它打标签。在这种情况下，尽管该模型甚至没有在音频上进行过训练，它却能够生成关于个人信息、家庭住址、位置数据、出行信息和目的地的正确信息。

英文原文

So here's a case study of a voice agent. In this case it was a GMO 4 E4B. Our baseline was 72 ordinary clips and spoken four voices. We're looking at 16 token pages. Obviously this granular granularity could change but here we just use shorter pages so we can have more data in the baseline. And then on the test we have different voices and one clean voice and the other one has a hidden instruction within it. And as you can as is shown here not only you can detect these cases just using simple anomaly detection but also you would know which part of um which page which part of your request had this anomalous event. We even went further and we trained a model that uses the KV pages as features and we didn't try to explain it or tag it with things that are useful. In this case, although it wasn't even trained on audio is able to generate correct information about personal information, home address, location data, travel information, and destination.

现在回到文本智能体。在这个案例中，是一个 Qwen 3 14B Live 在 L11 Cache 之下。左边的图向你展示了攻击流量与正常流量通常是如何相互区分的概念。而右边可能更有意义的那张图，使用的是 Hugging Face 上的 AgentBench。它拥有由良性智能体轨迹构成的自身基线，以及八种智能体故障，从注入的工具描述到错误信息等等。我们观察了异常检测会生成的 Top 10 警报，其中大约有 7 到 8 次这些警报实际上都是有意义且有用的异常活动。我们甚至更进一步，探究了是否存在任何比其余事件更为突出的异常事件类型。

英文原文

So now back to text agents. In this case, it was a quan 3 14 billion live and below 11 cache. The one on the left shows you an idea for how attacks and normal traffic would typically separate from each other. And then the one on the right, which is perhaps more meaningful, it was using the at bench and hugging face. It has its own baseline of benign agent traces and eight kind of agent failure from injected tools description to misinformation and so forth. We look at the top 10 alerts that anomalous anomaly detection would generate and about seven to eight times of out of 10 of these alerts are actually anomalous meaningful and useful anomalous activities. We even went further and we looked at whether there's any kind of anomalous events that stand out more than the rest.

呃，有两点让生产环境下的事情变得困难。首先是模型的变动。不仅有微调和新版本发布，还有每个运行在不同模型上的智能体，以及为每个请求挑选专长模型的路由器。这意味着每个模型都会以自己的坐标系来写入它的 Page。所以要么你去学习每一个基线、分类器和解释器，这在计算上是极其昂贵的；要么你就必须找到一种方法，将它们全部映射到同一个共享的表示空间中。

英文原文

Um, two make things hard in production. First, models change. fine tunes and new releases, but also agents that each run on a different model and routers that pick a specialist model per request. That means each model writes its pages its own coordinates. So either you learn every baseline, classifier, and explainer, which is computationally prohibitive, or you have to find a way to map them all into one shared representation space.

其次，实时意味着要搬运大对象。例如，Qwen 3 14B 的一个 1256-token 块大约对应 40MB 的 KV。压缩和流式传输（比如 HashGen）可能会有所帮助，但为了安全分析而搬运这些数据并在实时中采取行动依然很慢，而这正是我们在……所解决的问题。

英文原文

Second, real time means moving big objects. For example, a 1256 token chunk of coin 314 billion is about 40 megabytes of KV compression and the streaming like hashgen could help, but moving that for security analysis and acting in real times is still a slow and that's what we solve at

在人工智能领域。我们正在寻找构建自己的推理技术栈，并将安全视为根本性设计决策的团队。希望这能对大家有所帮助。非常感谢大家的聆听。

英文原文

in AI. We are looking for teams building their own inferences stacks who treat security as a foundational design decision. I hope this was helpful for you. Thank you so much for listening.

加油。热度。热度。热度。热度。嘿，嘿，嘿。这里气氛热起来了。

英文原文

Come on. Heat. Heat. Heat. Heat. Hey, hey, hey. Heat up here.

在我们进入压轴闭幕主题演讲之际，我们很高兴能请回一位熟悉的面孔，带领我们完成最后几场环节。女士们先生们，请热烈欢迎我们的主持人——DuckBill 的首席云经济学家，Corey Quinn。

英文原文

As we head into our closing keynotes, we're excited to bring back a familiar face to guide us through the final sessions. Ladies and gentlemen, please welcome back our MC, chief cloud economist at Duck Bill, Corey Quinn.

我们坚持到了最后。欢迎大家回来。希望大家度过了充实的一天，收获了精彩的演讲、想法、交流、结识了新朋友，可能还有新的劲敌，而且据说在展区那边甚至还有人切磋了一点《任天堂明星大乱斗》。呃，我曾经是、现在是、将来也依然是 Corey Quinn，在过去几天里能和大家一起参加 2026 年 AI 工程师代码峰会，感觉真是棒极了。

英文原文

We made it. Welcome back everyone. Hope you've had a great day of talks, ideas, conversations, new friends, new nemesis potentially, and supposedly even a little bit of Smash Brothers over in the expo. Uh I'm am have been and will remain Corey Quinn, and it has been terrific to be here with you all the last few days here at the AI Engineer Code Summit 2026.

呃，在我们最后一次深入展开晚间议程之前，我想借此机会向让本次活动成为可能的合作伙伴致谢，首先是主办赞助商 Google DeepMind。稍后会带来更多关于他们的介绍。当然还有休息室赞助商和白金赞助商、黄金赞助商、白银赞助商，呃，以及所有支持性合作伙伴，他们自己虽然没有获得特定级别的奖牌或颜色，但我们对他们全部都深表感谢。让我们为他们所有人热烈鼓掌。

英文原文

Uh before we dive into the evening programming for the last time, I want to take that moment to recognize the partners who make this event possible with presenting sponsor Google DeepMind. More on them in a second. And of course the lounge and platinum sponsors, the gold sponsors, the silver sponsors, uh and all of the supporting partners as well who do not themselves get a medal or color, but we appreciate them all. Let's get a round of applause for all of them.

[掌声]

英文原文

>> [applause]

Google DeepMind 主题演讲：Gemini 4 后训练

那么现在，让我们对接下来的压轴晚间主题演讲议程充满期待，以高潮为本次大会画上圆满句号。作为最后这个环节的第一场主题演讲，我们将听到来自 Google DeepMind 的分享。呃，与外面的所有大公司不同，不知怎么的，他们变得非常擅长给事物命名。我非常喜欢他们的模型命名法。他们现在正用元素来命名模型：Argon（氩）、Carbon（碳）、Hydrogen（氢），以及 Surprise（惊讶）。

英文原文

>> So now let's get excited for the final evening of keynote programming here and bring the show to a close on a high note. For the first keynote of the final block, we're going to be hearing from Google DeepMind. Uh they unlike every large company out there have gotten good at naming things somehow. I am loving their model nomenclature. They're naming them after elements now. Argon, carbon, hydrogen, and surprise.

噢，拜托。这明明是个不错的冷笑话。啊，来了。好的，能听出大家终于反应过来了。是的。呃，接下来登台为我们讲解他们如何针对长视野智能体编程进行后训练的讲者，请大家热烈欢迎 Google DeepMind 副总裁兼 Gemini 后训练负责人 Melvin Johnson。

英文原文

Oh, come on. That was a good dad joke. Ah, there we go. Okay, you can hear the penny dropping. Yeah. Uh, here to wind up talking about how they do postraining for long horizon agentic coding. Please give a warm welcome to vice president of Gemini postraining lead at Google DeepMind, Melvin Johnson.

[掌声]

英文原文

[applause]

大家好，很高兴来到这里。我是 Melvin，负责 Gemini 的后训练工作。所以今天我想带大家回顾一段历程，看看我们是如何对 Gemini 4 进行后训练的。而我们的历程始于人们说我们不再是前沿模型。

英文原文

Hey everyone, good to be here. I'm Melvin and I lead post training for Gemini. So today I thought I'll take you through a little journey of how we post train Gemini 4. And our journey begins by people calling us not a frontier model.

嗯，这里有一个例子。大家都知道，有人说“Gemini 玩完了（cooked），但 GCP 正在大显身手（cooking）”。从各种实际意义上来说，我们认为 DeepMind 已不再是一家前沿实验室。我们非常尊重的 Ethan Mollick，他用自己委婉的方式说，两次发布之间隔的时间相当长，这就是他对我们的评价。

英文原文

Um so here's an example. You know people say Gemini is cooked but GCP is cooking. For all intents and purposes we believe DeepMind is no longer a frontier lab. Ethan Malik who we respect a lot. He says in his own nice way pretty long gap between launches is what he calls us.

而我今天来到这里是要告诉大家，我们并没有消亡。我们依然在场。呃，我们已经发布了呃 Gemini 4 Argon。我们宣布了 Carbon，并且发布了 Carbon，这一切全都在过去的 28 天之内。所以我们正处于，大家知道，我们正处于迅猛推进的状态。而且还不止于此。

英文原文

And I'm here to tell you we're not dead. We're here. Uh and we have launched uh Gemini 4 Argon. We announced Carbon and we launched Carbon all within the last 28 days. So we're on a we're you know we're on a run here. And that's not it.

呃，在这个过程中，我们实际上赢得了来自 Dylan 的道歉。不，他说“正式向谷歌道歉”。嗯，“他们实际上做得很出色（cooked），并没有完蛋（not cooked）”。嗯，Ethan 说，大家知道，Gemini 4 最显著的方面之一就是我们推出新产品的速度。

英文原文

Uh in the process we actually garnered an apology from Dylan. No, he says offic official apology to Google Google. Um they actually cooked they're not cooked. Um Ethan says you know one of the most significant aspects of Gemini 4 is the speed at which we're launching things.

嗯，所以我们曾经经历了 236 天没有推出前沿模型，而我们在 28 天内推出了两款前沿模型。这相当于我们发版速度提升了 10 倍。而且大家知道，人们以前称我们“谷歌差不多完蛋了”，但谷歌并没有完蛋。所以，很高兴嗯能看到那种情绪风向转变。能站在逆转后的这一边让人高兴。其实能站在接收负面评论的一方也是一件好事，因为这会鞭策你做得更好。

英文原文

Um so we've gone 236 days without having a Frontier model and we've had two Frontier models in 28 days. That's a 10x improvement in our speed of shipping. And you know, people call us Google kind of cooked, but Google's not cooked. So, it's good um to see see that sentiment. It's glad to be on the other end. It's also good to be on the receiving end of the bad comments because it pushes you to do better.

我想先播放一小段视频，展示一下呃 Carbon 构建的一系列成果。如果我们能播放视频的话，麻烦了。这就来了。退回去。来了。能放了吗？不行。让我看看换到笔记本电脑上能不能解决。谢谢。

英文原文

>> I want to start by playing a little video of a bunch of things that uh Carvin has built. If we can cue the video, please. There we go. Go back. There we go. Did that work? No. Let me see if going to the laptop is going to fix it. Thank you.

[掌声]

英文原文

[applause]

现在，我们要向模型自己构建出那段视频致敬。大家看到了很多非常棒的例子，展示了该模型在使用 Blender 构建 3D 模型资产方面表现得好得多。里面有很多游戏。真实世界的用例、模型处理、多模态文档。呃，但是大家知道，这不仅仅是呃演示。Carbon 实际上在大量真实世界的客户用例中排名第一。

英文原文

Now kudos to the model for building that video uh itself. So you saw great examples of the model doing much better at 3D model building assets using Blender. There's lots of games in there. The real world use cases, model processing, multimodel documents. Uh but you know, it's not just the uh the demos. Carbon's actually number one on a bunch of real world customer use cases.

而且我们非常兴奋的一点在于，大家知道，它不仅呃在这些基准测试上非常出色，而且实际上便宜得多。嗯，在 Cursor Bench 上，它是当今排名第一的模型，而且比排名第二的模型便宜 3.3 倍。在 Perplexity 的 Wander 上——这是一个针对深度研究嗯查询的基准测试——它是表现最好的模型。大家知道，你们看到嗯竞争对手的模型发布出来，我们在该基准测试上比第二好的模型便宜 9 倍；而在 Automation Bench——也就是 Zapier 的呃基准测试上——我们再次夺得第一，同时每项任务便宜 2 倍嗯。

英文原文

And one of the things that we're really excited about is, you know, it's not just uh really good on these benchmarks, it's actually much cheaper. Um so on Cursor Bench, it's the number one model today and it's 3.3 times cheaper than the second best model. On Perplexity's Wander, which is a benchmark for deep research um queries, it's the best model. you know you saw um competitor models come out were 9x cheaper than the second best model there and on automation bench which is Zapier's uh benchmark again were number one while being two times um cheaper per task

而且还不止这些，我是说今天我们宣布了我们 Gemini 4 系列的新元素呃，并且它不是 Surprise（惊讶），呃而是氢（Hydrogen）。

英文原文

and that's not it I mean today we announced the new element uh to our Gemini 4 series and it's not surprise uh it's uh hydrogen

呃，大家可能在今天早些时候看到了发布消息。Gemini 4 Hydrogen 正是我们试图将 Gemini 4 的智能普及给所有人的载体。它快速、可靠，已经在所有平台上推出：在 Antigravity 上、在 Gemini API 上、在 Gemini 应用中，以及在搜索中。嗯，大家知道，我这里有一张图表，展示了许多人可能熟悉的 AA 指数，x 轴是时间。大家可以看到，在同等智能水平下，它是速度最快的模型之一。

英文原文

uh So, you know, you might have seen the launch earlier today. Gemini 4 hydrogen is where we're trying to bring the Gemini 4 intelligence to everyone. It's fast, it's dependable, it's launched on all surfaces on anti-gravity, on the Gemini API, in the Gemini app, and on search. Um, and you know, I have a graph that shows the AA index with many of you might be familiar with, and time on the x-axis. You can see it's one of the fastest models for that level of intelligence.

为了让大家了解 Gemini 已经走了多远，大家知道，3.8 Flash 在 Terminal Bench 4 上的成绩大约是 20%。而 Hydrogen 大约是 60%。所以我们的性能提升了三倍，同时速度也快了三倍。对吧？所以这绝非易事，而且我们大家知道真的很高兴能把这个成果交到你们手中，让大家可以用它来构建应用。

英文原文

And to give you a sense of how far you know Gemini has come 3.8 flash on terminal bench 4 was about 20%. And hydrogen is about 60%. So we're three times better performant while being three times faster. Right? So that's no easy task and we're you know really happy to bring that to you in your hands so you can build things with it.

这就补全了大家知道的我们呃 Gemini 4 系列。我们有 Gemini 4 Carbon，用于深度前沿智能的前沿模型；Gemini 4 Hydrogen，它速度快，用于日常呃执行。我们还会有更多的元素。大家知道，元素周期表上当然还剩下很多元素，我们还有很多工作要做，才能把它们用尽。呃，但我们很高兴呃能把它发布出来，供大家测试，供大家上手体验。

英文原文

And that completes you know our uh Gemini 4 series. So we have Gemini 4 carbon a frontier model for deep frontier intelligence Gemini 4 hydrogen which is fast for everyday uh execution. And we'll have more elements. You know, of course, there are many elements left in the periodic table and we have a lot of work to do to exhaust them. Uh but we're glad uh to put it out there for you all to test it and for you all to play with it. Um

后训练的工程挑战与闭环

嗯，现在我想关于模型的市场宣传已经足够了。我想，有人告诉我台下是非常懂技术的受众。因此我想稍微谈谈为什么对这些前沿模型进行后训练是困难的，以及这其中涉及哪些环节。首先我想说，大家知道，当今在谷歌，AI 编写了大约 75% 的新增代码。而这在几年前还不到 25%。并且每天在谷歌内部有 100,000 人使用 Antigravity 编写代码。

英文原文

now I think enough marketing with the models. I want to I was told this is a deeply technical audience. So I want to talk a little bit about why post- training these frontier models is hard and what goes into that. Let me start by saying, you know, AI writes about 75% of Google's new code today. And this is up from less than 25% a few years ago. And there are 100,000 people using anti-gravity writing code inside Google every single day.

如今，大家所执行的任务的复杂度在过去几年中也呈指数级增长。我们最初是从大家知道的模型单纯进行聊天补全开始的，那只需要花费几秒钟，发展到在几分钟内编写小型函数。大家可以提交 PR 并解决一个 issue，大约需要说一小时左右，但现在人们正在利用该模型执行长达数天的真正工程任务。人们拥有智能体群（swarms of agents），利用该模型解决真正的开放性问题。在整个谷歌，我们的工程师正在使用 Antigravity 和我们的 Gemini 4 模型来跨越数天解决任务。

英文原文

Now, the complexity of the tasks that people do have also increased exponentially in the last couple of years. We started with you know models just completing chats which took seconds to writing small functions within the minutes. You can submit a PR and resolve an issue takes about an hour let's say but now people are doing days of real engineering tasks with the model. People have swarms of agents solving real open problems with the model. We have engineers across Google using anti-gravity and our Gemini 4 models to solve tasks over multiple days.

那么，我们究竟如何对 Gemini 进行后训练以应对真正的工程任务呢？我们必须从高难度任务入手。因此，我们确实非常需要为模型搜集难题。对这些模型进行评分实际上是一个非常困难的问题。所以，你希望像一位经验丰富的代码审查员那样去打分。你希望像一位经验丰富的工程师评审解决方案那样去打分。大规模强化学习（RL）就是我们训练模型的方式。同时我们希望在拥有 100,000 名工程师的社区中对我们的模型进行内部试用（dogfood），并循环往复。这样，内部试用中的任何失败情况都会成为下一次循环迭代的任务。

英文原文

Now, how do we go about post- training Gemini for real engineering tasks? We have to start with hard tasks. So, we need really need to source hard problems for the model. Grading of these models is actually a very hard problem. So, you want to grade like an experienced code reviewer would grade. You want to grade like an experienced engineer would grade a solution. Large scale RL is how we train the model. And we want to dog food our models with our 100,000 engineer community and repeat the loop. So any failures in the dog food become tasks for the next iteration of the loop.

所以我想先对这些部分的每一个做更进一步的详细介绍。那么让我们从高难度任务开始，对吧？嗯，大家可能听说过，随着模型变得越来越好，许多基准测试都趋于饱和，而且不仅基准测试趋于饱和，我们的训练数据也同样趋于饱和。这里有一张图表，展示了针对 Gemini 4 模型的三个不同迭代版本，可学习提示词（learnable prompts）的份额下降了多少。比方说我们在第一次迭代中有 x% 的可学习提示词。到我们完成并训练下一个版本的时候，这些提示词就已经太简单了

英文原文

So I want to start by going into a little bit more detail of each of these sections. So let's start with the hard tasks, right? Um so you might have heard you know as models are getting better lots of benchmarks are getting saturated and it's not just the benchmarks that get saturated our training data gets saturated too. So here is a graph that shows for three different iterations of the Gemini 4 models how much uh the share of learnable prompts has gone down. So let's say we had x% of learnable prompts for the first iteration. By the time we finish and train the next one the prompts are too easy

困难任务与评分标准

……针对第二个模型。我们完成那个之后，接着训练第三个模型。对第三个模型来说，这些 prompt 太简单了。那么在简单的任务上进行训练有什么问题呢？简单的任务给你带来的学习信号是零，对吧？也就是说，你给出一个 prompt，所有的 rollout 都会是正确的。这里面没有方差，没有任何强化学习信号。所以你真的需要专注于获取困难任务，而且你获取困难任务的速度真的必须快于模型变强的速度，对吧？

英文原文

for the second one. We finish that and train the third one. The prompts are too easy for the third one. So what's the problem with training on easy task? Easy task give you zero learning signal, right? So you have a prompt when all of your rollouts are going to be correct. There's no variance. There's no RL signal. So you really need to focus on getting hard tasks and you really need to be faster at getting hard tasks than the models are getting better, right?

嗯，那么我们该怎么做呢？我们首先得识别出这些简单的任务并修剪掉它们。因此，我们从数据混合中剔除这些对模型来说太简单的简单 band 风格任务。我们拓展到难度大得多的 suite 任务中，例如多文件代码库。我们从单元测试或二元正确性转变为基于评分标准（rubric）的评分。评分标准能给你密集的信号，它们不仅仅是 0 或 1，在评估准则内部可以有更多的梯度。

英文原文

Um so how do we go about it? We first have to you know identify these easy tasks and prune them. So we cut these simple band style tasks from the mixture that are too easy for the models. We expand into much harder sweet tasks for example that are multi-file repos. We go from unit test or binary correctness um to rubric based grading. And rubrics give you dense signal. They're not just 01. there can be um more gradients within the um within the criteria.

这里有一些我们用来训练的困难任务的例子。我们研究多文件代码库，并从中提取 commit。模型会自动从这些 commit 中提取出评分标准。我们实际上并不强制要求涉及单元测试，我们完全可以基于这些代码库动态生成评分标准。另一个例子，一个绝佳的任务例子是：尝试教模型从零构建 CLI 工具。你从一个空文件夹和一份规格说明（specification）开始。规格说明上写着：嘿，你必须构建这个工具，它叫 GP，这些是 GP 的参数，这些是 flag，这是具体功能。然后你让模型从零构建出整个工具。因为你知道 GRP 的功能会是什么样，所以你拥有一个隐藏的测试套件，可以在上面对生成的二进制文件进行评估。最后一个例子是生产代码库的 diff。你会去观察模型在生产代码库内部的真实使用情况，并从中提取出有用的任务。

英文原文

Here are some examples of hot tasks that we train on. So uh we look at multifile repos and we extract commits from them. From those commits we have models automatically extract rubrics. Um we don't actually ne necessitate unit tests being involved. We can actually uh generate rubrics on the fly based on these repos. Another example, a great example of tasks are you try to teach the model to build uh CLI tools from scratch. So you start with an empty folder and a specification. The specification says, hey, you have to build this tool. It's called GP. These are the parameters of GP. These are the flags. This is the functionality. And you let the model build the entire tool from scratch. And you know what the functionality for GRP is going to be. So you have a hidden test suite that you can evaluate the binary on. The last uh example is production codebased diffs. So you look at real usage of the model inside your production codebase and extract useful tasks out of them.

防范作弊与裁判系统加固

现在让我来稍微谈一下评分裁判（grader）。

智能体，你知道的，因为你是在训练它们通过测试，所以它们会想方设法去通过测试。但即使测试通过了，也并不一定意味着它们真正解决了问题。有些例子里，智能体会绕过检查器；有些例子里，智能体会钻 benchmark 的漏洞；还有些例子里，智能体会伪造证据。你知道的，它把伪造的证据写入一个临时文件，然后删除任何表明它伪造证据的痕迹，对吧？所以，随着这些模型变得越来越强大，你必须用更聪明的方式来评判它们。

英文原文

Now let me talk a little bit about uh the graders.

So agents you know they find any way you're you're training them to pass the test. So they will find any way to pass the test. But if the test pass, that doesn't necessarily mean that they've solved the problem. There are examples where the agent will bypass the checker. There are examples where the agent will game the benchmark. There are examples where the agent will fake the evidence. You know, it writes the uh fake evidence into a temp file. It'll delete any evidence of it faking it, right? Um, so you have to be smarter about how you judge these models, especially as they become more capable.

这就是我所说的“像代码审查者一样评分”。如果你做过代码审查，面对一个非常严格的审查者，你会知道他们不会只考虑方案的正确性。他们实际上还会考虑代码风格、可维护性，以及许多不同的方面。所以我们所做的，就是从基于单元测试的正确性转变为基于评分标准，而这些标准是根据真实 commit 生成的。评分标准还能提供密集的学习信号，而不仅仅是 0 或 1 的信号，并且我们会惩罚任何试图投机取巧走捷径的方案，对吧？这是一个评分标准的例子，其中的评估准则是：需要通过哪些边缘情况、需要维护哪种代码规范整洁度、需要满足哪些关于验证的约束条件。

英文原文

Now this is what I meant by grading like a reviewer. So if you've done a code review, you have a very strict reviewer, you're going to know they're not just going to think about the correctness of the solution. They're actually going to think about the style. They're going to think about maintainability. They're going to think of many different aspects. So what we do is we move from unit test based correctness to rubric base and these are generated from real commits. The rubrics also provide dense learning signal not just 01 signal and we penalize any sort of solution that is looking for a workaround. Right? So here is an example of a rubric where the criteria is you know here are the edge cases that need to be passed. This is an example of hygiene that needs to be maintained. These are constraints around verification that need to be satisfied.

现在，这不仅仅关乎评分所需的标准。我们还必须警惕评分裁判本身被钻空子，你真的需要加固裁判所处的环境。因此，我们考虑将评分裁判置于隔离的沙箱中。我们移除了 Git 历史和 Git remote，这样答案就不会泄露。测试集是保留不公开的。只有当智能体完成并提交了解决方案时，我们才会真正揭晓测试。我们遇到过这样的情况：智能体先假装说“嘿，我完成了”，等到测试显现后，它又去看测试并重新提交解决方案，对吧？所以你不希望发生这种事。此外，我们也希望在打分时做得更加细致。我们会提取完整的 diff，参考基准，并考量多个方面。不仅仅是正确性，我们还要看代码质量，还要看所做改动的范围。最后，我们要惩罚智能体的任何作弊企图，同时又注意不去管制它的思维链（thoughts）。任何作弊或破坏环境的行为都会受到非常严厉的强化学习负奖励。而所有这些都仅仅是根据模型采取的行动来检测的，而不是根据思维。在处理作弊方面，一个很好的例子是：在 Terminal Bench 4 上，Carbon 的作弊率为 0%。我们之前的模型作弊率要高得多，并且对抗性大约是 Argon 的五分之一。所以对于模型的每一代，我们都在努力让它更加对齐，让它不会试图去作弊。

英文原文

Now, it's not just about the rubrics needed for grading. We also need to be careful about the graders not being gamed and you really need to harden the environment that the grader is in. So, we think about putting the graders in a isolated sandbox. So we remove git history and git remote so the answers don't leak. The test sets are held out. So only when the agent is done and submits the solution do we actually reveal the test. We've had instances where the agent will fake it and say hey I I'm done and the tests come into play and then it'll look at the test and resubmit the solution. Right? So you don't want that. Um we also want to be much more detailed with our scoring. So we take the full diff and we look at the reference and we look at multiple aspects. It's not just the correctness. We also want to look at the code quality. We also want to look at the scope uh of the change that was done. And then finally, we want to penalize any sort of cheating attempt by the agent while being careful not to police the thoughts. Um so any you know cheating or hampering of the environment gets a very negative RL reward. Uh and all of this is detected just based on the actions that the model is taking, not the thoughts. Um so a good example of how we've handled cheating is you know carbon for instance has 0% cheating rate on terminal bench 4. Our previous models had a much higher cheating rate and it's about 5x less adversarial than argon. So every generation of the model we're trying to make it much more aligned so it's not trying to cheat.

模型诚实性与虚构抑制

现在最后，我想稍微谈谈模型行为，具体来说是模型诚实性。

智能体总是想表现得很有用，有时它们会假装有用。你在使用智能体时可能见过这种情况：有时它们声称测试已经通过，但显然并没有通过；它们宣布已完成，但显然没有完成；它们声称自己做了修改，但并没有做那些修改；或者它们会凭空捏造事实、API 等等；或者它们会给你一大篇包含大量数字的长篇大论，让你根本无法验证，对吧？我们经历过所有这些情况。

英文原文

Now finally I think uh I want to talk about model behavior a little bit and modeling honesty specifically.

Now agents they always want to be helpful and sometimes they fake helpfulness. You might have seen this in your usage of the agents. So sometimes they'll claim that tests have passed when they've obviously not passed. They'll declare done but obviously they're not done. They'll make edits that they'll claim to have made edits but they've not made those edits. Or they'll invent facts and APIs and whatnot. or they will give you a wall of text that has so many numbers that is going to be impossible for you to verify. Right? We've gone through all of these examples.

这是 Gemini 中捏造率的一个例子。在一次内部评估中，我们在那项困难的评测中一开始有 8% 的捏造率。在 Hydrogen 上，我们大幅将其削减到大约 3%，而在 Carbon 上，这一比例又减半，降到了大约 1.5%。但我们正努力将这个数字降至零。就像你绝不希望自己的智能体捏造任何东西一样，对吧？但它们之所以试图捏造，是因为它们想提供帮助。你真的必须下大功夫去对齐它们，让它们提供帮助，但不是通过弄虚作假的方式。

英文原文

Now, here is an example of fabrication rates in Gemini. Right? So, we on an internal eval started with 8% fabrication in that hard eval. We cut that down dramatically to about 3% for hydrogen and carbon is half of that is about 1.5%. But we're striving to get this down to zero. Like you don't want your agents fabricating anything, right? But they try to fabricate because they want to be helpful. And you really have to try hard to align them to be helpful but not by faking it.

我们如何减少捏造？不管你信不信，你必须清洗你的数据，对吧。所以我们过滤掉了大量训练轨迹，在这些轨迹中，不仅存在有意的捏造，也存在无意的捏造。其中有许多未得到充分支撑的主张，它们没有经过验证。我们以完整轨迹为依据（grounding）。如果智能体说“我已完成任务”并给出了最终回复，我们实际上会去查看那个回复，查看智能体在轨迹中为达成该最终解决方案所采取的所有动作，并审视所采取的步骤。因此，当智能体说“我完成了”并给你一份总结时，必须以所采取的实际动作为依据。并且我们希望奖励诚实。如果有难题，智能体更好的做法是说：“嘿，这是我的部分进展。这是一个难题。我在这里停下了。这是我的局部解。”而不是在最后一步弄虚作假，说：“嘿，我已经解决了这个问题。你看，搞定了。”

英文原文

How do we reduce the fabrication? Believe it or not, you have to clean your data. Uh right. So we filter out lots of training trajectories where there is you know not just uh intended fabrication but unintended fabrication. There's lots of claims that are not fully supported. They're not verified. We ground in the full trajectory. So if the agent says I've completed the task and it has a final response, we actually look at that response and see all of the actions that the agent had taken in the trajectory to get to that final solution and we look at the steps that were taken. So you really need to when the agent says I'm done and gives you a summary that needs to be grounded in the actions that were taken. And we want to reward honesty. So if there's a hard problem, it's better for the agent to say, "Hey, this is my partial progress. It's a hard problem. I'm done here. This is my partial solution." Rather than faking the last step and saying, "Hey, I've solved the problem. You know, this is done."

以周为周期的快速迭代发布

最后，在几周内完成发布。那么我们为什么能够如此迅速地发布呢？

其中最大的原因，是我们在 Google 内部拥有一支超过 10 万名工程师的团队，他们将该模型用于实际任务。你们很多人可能知道 Google 采用的是单一代码库（monorepo）。什么是 monorepo？整家公司共用同一个代码库，对吧，我不知道该怎么衡量这个，但我想它可能是世界上最大的单一代码库之一。那是一个非常复杂的代码库，而我们拥有世界上最优秀的人才，使用着一个极其复杂的代码库来完成极其复杂的任务。因此，每次我们通过后训练得到新的候选模型时，我们实际上都会将该模型提供给我们的用户。我们通过多个不同的候选模型与他们进行盲测对决（blind battles）。我们让用户选出他们想要的最佳模型。我们也会运行 Elis，在底层将人员路由到实验性模型，并从中获取信号和反馈，然后把所有这些信号和阻力反馈到下一个候选模型的训练中。

所以这是一个帮助我们快速迭代、真正针对困难问题进行训练的循环，并且随着……

英文原文

Finally, shipping in weeks. So how are we able to ship so rapidly and the biggest reason for that is our internal uh team of 100,000 plus engineers at Google using the model for realistic tasks. You know many of you might know Google is a monor repo and what is a monor repo? The entire company is one codebase right and I don't know how to measure this but I think it might be one of the largest monor repos in the world. Now that's a really complex uh code base and we have the best people in the world using a really complex codebase to do really complex tasks. So every time we have a new candidate from post training, we actually serve that model uh for our users. We run blind battles with them through multiple different candidates. We let users you know uh pick the best model that they want. We also run Elis where under the hood we're routing the routing people to experimental models and get signal and feedback from that and we take all of these signals and friction back into training the next candidate.

So this is a cycle that helps us you know rapidly iterate really train on hard problems and as

随着模型变得越来越好，工程师们将会以各种有趣的方式去推动模型。

英文原文

the models get better the engineers are going to push the model in interesting ways.

这也是为什么，你看，我们已经从数周发布……呃，从按季度发布变成了按数周发布。在 Gemini 3 时代，我们每两到三个月发布一次，但现在我们在过去 16 周内已经发布了 6 次，对吧？我们非常兴奋能让大家试用 Carbon 和 Hydrogen，呃，还有 Gemini 4。

英文原文

This is also why, you know, we've gone from releasing in weeks um to releasing to releasing in quarters to releasing in weeks. So, in the three in the Gemini 3 era, we've had launches every two to three months, but now we've had six releases in the last 16 weeks, right? Uh and we're very excited to have you all try Carbon and Hydrogen uh and Gemini 4.

呃，我们也很期待看到大家用它做出什么成果。呃，我们还没有结束。所以，我们还有很多元素要去一一探索。呃，就说到这里，谢谢大家。

[掌声]

英文原文

Um, and we're excited for for for us to see what you all do with it. Uh, and we're not done. So, we have a lot more elements uh to go through. Uh, with that, you know, thank you.

[applause]

热度。热度。

从帮助开发者理解复杂代码库，到打造下一代 AI 编程智能体，我们的下一位演讲者一直处于开发者工具的最前沿。女士们，先生们，请热烈欢迎 AMP 联合创始人兼首席执行官 Quinn Slack。

英文原文

Heat. Heat.

From helping developers navigate complex code bases to building the next generation of AI coding agents, our next speaker has been at the forefront of developer tools. Ladies and gentlemen, please welcome Quinn Slack, co-founder and CEO of AMP.

Quinn Slack 演讲：智能体如何重塑软件构建

大家好，我是 Quinn。呃，很高兴见到大家。你们可能想往前坐一点，因为就像 NBA 比赛一样，我带了一些周边和帽子要发给大家。所以，如果你坐在后排而且喜欢帽子的话，往前走走。呃，好的。

英文原文

Hey y'all, I'm Quinn. Uh, great to see you. You might want to come up in front a little more because I do, it's like an NBA game, have some swag and hats to give out. So, if you're in the back and if you like hats, come on up. Um, all right.

不过不管怎么说，我叫 Quinn Slack。我是 AMP 的创始人之一，之前在 Sourcegraph。不过今天我不想谈 AMP。我想讲一个真正的个人故事：智能体如何深深震撼了我，并改变了我的生活。曾经有那么一天，我走出室外，鸟儿叫得更欢了，天空也显得更蓝了；以及它们如何改变了我的公司，以及我认为它们将如何改变每一家公司。有时候看到朋友、我在乎的人、以及我在乎的公司在这个问题上挣扎，我感到挺受挫的。所以，我们接下来要体验一段跌宕起伏的过程，不过，呃，要不我们这就开始吧？我知道快到晚饭时间了。

英文原文

As whatever, you know, that said, um, my name is Quinn Slack. I am one of the founders of AMP and had been at SourceCraft before. I don't want to talk about AMP though. I want to talk about really a personal story of how agents blew my mind and changed my life. There was a day when I walked outside and the birds were chirping more and the sky was bluer and how they changed my company and how I think they're going to change every company. And it's kind of frustrating sometimes to see friends and people I care about, companies I care about, really struggle with this. So, we're going to go on a little bit of a roller coaster, but uh why don't we get started? I know it's almost dinner time.

呃，有人想要帽子吗？一些 AMP 帽子。好的，来了。我看到有人走近了。接住了。好的。

希望能别砸到人。

接住了。

还有那边。

好的。

英文原文

Uh, anyone want some hats? Some AMP hats. All right, here we go. I see people getting closer. There we go. All right.

Hope I don't hit anyone.

There we go.

And over there.

All right.

对，每个人都能拿到帽子。每个人都有份。活动结束后过来找我。

英文原文

>> Yeah, everybody gets a hat. Everybody gets a hat. come up and find me after.

好的。呃，在 2025 年 1 月有那么一个瞬间，当时我正在写代码，我想看看如果直接做个智能体会怎样。那还是在 Claude Code 出现之前，做的东西可以说是世界上最蠢的事情，比如改一下它的背景颜色。但是当我第一次看到智能体真正运转起来时，整个人都被震撼了。它能运行命令，它能自我检查，它能给自己建立反馈循环。这太让我震撼了。而且我当时产生了一种感觉，我想每个人第一次见到智能体时都会有这种感觉：在那之后，一切都将彻底改变。

英文原文

All right. Uh there was a moment in January of 2025 when I was just coding and I wanted to see what if you just make an agent. This was before Claude code and it was like the dumbest stuff ever. It was like change its background color. But it just blew my mind when I saw an agent actually work for the first time. It ran commands. It looked at itself. It gave itself its own feedback loops. That blew my mind. And I had this sense I think that everyone has when they first see an agent that everything is going to change after that.

关于构建软件的一切都将彻底改变。我想我们都感受到了，关于为了做软件而组建公司的方式、销售软件的方式、分发软件的方式，一切都将彻底改变。所有其他工作都将发生变化。那绝对是一个疯狂、疯狂的时刻。呃，当时我和我们团队的 Thorsten Ball 正好在法兰克福机场。呃，他写了那本关于编译器的书。认识我们团队 Thorsten 的人都知道。对，他是个很酷的家伙。呃，我们在法兰克福机场简直被震撼得说不出话来，我们深知一切都将改变。我们不知道这一切最终会走向何方，但我们就是想全速前进。我们只想迎头冲上去。因此，在 Sourcegraph 内部——那是一家 200 人的企业级软件公司，我是 CEO，我们做代码搜索——我们决定直接开始做这个，抛开其他所有事情，专心打造这个产品。不去参加销售拜访，不去开什么高管会议，统统都不参加。直接屏蔽掉所有的干扰。

英文原文

Everything is going to change about how you build software. I think we've all felt that everything is going to change about how you form companies to work on software and how you sell software, how you distribute software. All other work is going to change. And it was just an absolutely crazy crazy moment. Um I was actually at the Frankfurt airport with Thorston Ball on our team. Um he wrote that compiler book. anyone who knows Thorston on our team. Yeah, he's a cool guy. Um, we were just like minds blowing at the Frankfurt airport and we knew that everything was going to change. We didn't know where this was all going to go, but we just wanted to go. We just wanted to go run at it. So, inside of Source Graph, which was a 200 person enterprise software company, I was the CEO and we made code search. We decided to just start building this to tune out everything else and just start building this thing. Not go to sales calls, not go to like exec meeting, none of that stuff. Just tune out all the distractions.

而我当时所处的位置，我想很多人都会说：“哇，你看，我是公司里的软件开发人员，我想做出改变。我希望我的公司能更加积极拥抱 AI，但我做不到。”呃，你看，我是 CEO，理论上我可以改变任何事情。我可以调整团队。我可以宣布不需要提 PR 做代码审查，呃，不需要遵循我们过去的那套流程，但即便如此，这依然非常艰难。我们最终的做法是从公司带走了 20 个人，呃，依然在 Sourcegraph 内部成立了一个小团队，也就是 AMP 团队，呃，那就是我们的尝试。所以对于那些正在试图在公司内顶住压力推进变革、努力让环境对智能体更友好的人，我非常理解你们，我希望你们能从这里带走一些收获。呃，我知道这有多难，同时也对你们的 CEO 多一份理解。这对他们来说同样很艰难。这是一种非常困难的变化和转型。

英文原文

And I was in this position that I think a lot of people say, you know, wow, you know, I'm a software developer at a company and I want to change. I want my company to be more AI forward and I can't. Um, you know, I was a CEO and in theory I could change anything. I could change teams. I could say you don't need to get poll request code review. uh you don't need to you know do any of the other processes we used to do and yet it was very hard what we did is we ended up taking 20 people uh from the company and inside of source graph still we made a little team the AMP team and uh that's what we tried so for those of you who are trying to push back against your company to try to make it more agent friendly I have a lot of sympathy for you and I hope that you can take away some things from here um I know how hard it is and also have some empathy for your CEO too. It's hard for them, too. This is a really hard kind of change and transformation.

但我们很清楚，有些事情一定会改变。这一切现在看来可能有点显而易见，但在 2025 年、2025 年初那阵子，关于“企业级就绪”的概念——即你把这件事做好，然后就能建一支销售团队拿去卖——我们知道这种模式将会彻底消亡。你必须站在最前沿。如果你尝到了点甜头，然后就开始逐条对照企业级清单去打勾以取悦普通企业买家，你的产品大概 3 个月内就会被淘汰，整个游戏规则不得不改变。除了让其他公司的员工真正爱上你的产品之外，其他一切都无所谓了；而 CEO 手头的预算是无限的，所有过去的常规逻辑都不适用了。这一点我们心里清楚。只是我们当时并不完全清楚呃我们会遇到哪些挑战。

英文原文

But we knew that there were some things that were going to change. And this all seems kind of obvious now, but back in 2025, early 2025, this idea of enterprise ready. You do this thing and then you can build a sales team and go sell it. We knew that that was just going to totally die off. You had to be on the frontier. If you had, you know, a taste of success and then you started doing all the enterprise checklist to make the median enterprise buyer happy, your product was going to go obsolete in like 3 months and the whole game had to change. Nothing mattered other than making it so that the people at these other companies loved your product and the CEO, the budget was infinite and all bets were off. So, we knew that. We didn't quite know uh all the challenges we'd run into.

呃，另外，在智能体能够构建产品或开发功能的这个世界里，你到底该如何思考差异化？我的意思是，直到今天依然有人问：“那为什么 AMP 更好？为什么，你看，Claude Code 比 Codex 或其他什么更好？”这么多人都在苦苦求索一个答案，希望其形式是：“你看，是因为这三项功能。”比如，呃，AMP 拥有 Orbs，这是非常优秀的云端智能体，而且我们有多人协作功能。但疯狂的地方在于，你们都知道如果我们能做到这些并且证明了它可行，那我们的任何竞争对手也同样能做到。既然如此，为什么还要去玩那种游戏呢？比如说，构建和销售软件到底意味着什么？如果你连一个月后它会变成什么样都不知道，你为什么还要去招聘销售团队？为什么还要按传统方式去做任何事情？

英文原文

Um, also, you know, in this world where agents can build a product or build a feature, how do you even think about differentiation? I mean, to this day, there's people who say, well, why is AMP better? Why is, you know, claude code better than codeex or whatever. And so many people are begging to have the answer in the form of, well, it's these three features. You know, uh, AMP has orbs, which are really good cloud agents, and we have multiplayer. But the crazy thing is, you all know that if we can do that, and if we prove it, then any of our competitors could as well. So, you know, why even play that game? Like, what does it even mean to build and sell software? Why would you go and hire a sales team? Why would you go and do anything in the traditional way if you don't even know what it's going to look like in a month?

而且如果你甚至不知道该如何描述差异化，那就根本不存在可持续的差异化。呃，对于 AMP 来说，你看，大家最终都很喜欢它。呃，但你甚至可以看到，像呃 Tailwind 的作者 Adam Wathan，他很喜欢 AMP，但他说，你看，这属于那种你无法仅凭言语去理解的东西，呃，你看，在亲自使用它之前，你无法理解它到底好在哪里。你能想象拥有一个庞大的产品管理团队、设计团队、工程团队，他们却与实际客户如此脱节吗？如果这就是你推广产品最好的方式，整套公司架构运转起来真的会极其艰难。所以我们知道自己必须改变一切，我们对太多的事情进行了重新思考，因为可以说在无意之中，我们置身于一些非常奇怪的限制之下。

英文原文

And if you don't even know how to describe differentiation, there's no sustainable differentiation. Um, for AMP, you know, people end up loving it. Um, but you can even see like, um, you know, Adam Wan, who's the, uh, Tailwind guy, he loves AMP, but he's like, you know, it's one of those things where you can't understand it. uh you know it's you can't understand why it's good until you use it. Can you imagine having a whole product management team, a design team, an engineering team that's so disconnected from the actual customers, a whole company apparatus if that's the best way that you can market a product, it's really tough. So we knew that we just had to change everything and we we rethought so many things because kind of unintentionally we were in under some weird constraints.

我们决定将 AMP 剥离出来独立运作。所以对于那些在大公司里待过的人，比如一个小团队，呃，你看，你在公司里的一个小团队，你试图以不同的方式做事，你试图成为大初创公司里的内部创业团队，你可能也有过这样的幻想：干脆我们直接脱离出来创办一家新公司吧。而我们真的这么做了。我们在投资人和员工的利益上都处理得很周全，每个人在两家公司都有股份，但我们确实独立拆分出来了；而且在整整一年里，直到上周五，我们都不能招聘任何人，也不能授予期权。法律层面的拆分流程极其痛苦，我们被迫只能靠带出来的 21 个人勉力支撑。

英文原文

We decided to spin out AMP. So for those of you who've been at a company like a small uh you know you're on a small team at a company you're trying to do things different you're trying to be the startup within a startup you probably have these fantasies of let's just go off and start a new company well we actually did that we you know did the right thing by investors and employees and everyone has ownership in both but we actually spun out and for a year until last Friday we couldn't hire anyone we couldn't grant options the legal spin-off process was a huge pain and we were forced to make do with the 21 people that we brought

在这个竞争异常激烈的环境中，我们根本没有时间去走……呃，你看，我们没办法为了搞强制性的 Pull Request 代码审查而把步调放得那么慢。我们不能开会。我们不能让时间被那些需要耗费好几个月才能做出决定的企业客户给吸干。因此，我们被迫走向了这种截然不同的工作方式。而且我们还处在这样一种氛围中：团队里的每一个人都在公司拥有巨大的利益份额。而那种当你拥有 AI 时所获得的切身利益机制……

英文原文

We didn't have time in this crazy competitive environment to do uh you know we couldn't move as as slowly in order to do um you know mandatory poll requests. We couldn't have meetings. We couldn't have our time be sucked up by enterprises that would need to take you know many months to decide on something. And so we were forced into this radically different way of working. And we also had this environment where every single person on the team has a huge stake in the company. And that skin in the game that you get when AI

这就让你能以少得多的人手来运转。

英文原文

makes it so you can operate with way less people.

切身利益的绑定（Skin in the game），对优秀人才的灵魂而言，就像是一种精神层面的东西。它让他们在所做的每一件事上都表现得更出色，让他们成长得更快。这就是我们所见证的。呃，我认为当你看到一家在没有任何那类东西的情况下以这种方式运作的公司时，就很显而易见：像这样的公司将如何碾压其他所有人，以及未来的每家公司将如何看起来都更像那样。

英文原文

Skin in the game, it's like this spiritual thing for the soul of of great people. It makes them better at everything they do. It makes them grow faster. And so this is what we have seen. Uh this I think when you've seen a company that operates like this without any of that it's obvious how companies like this are going to crush everyone else and how every company in the future is going to look a lot more like that.

所以是的，我们把 AMP 拆分独立出来了，并且我们不得不去摸索出很多新的工作方式，来应对这些我们事先根本不知道会面临的限制；而且正如我所说的，我们当时无法招人。结果就是，顺便说一下这是一张对数图表，当该领域所有其他大竞争对手为了去利用这一惊人的 AI 技术而不断扩张并建立老式公司时，我们不得不保持在 21 个人。但那棒极了，因为那成了迫使我们以不同方式工作的一种强制机制，迫使我们去摸索：在无法招人的情况下，你怎样才能借助智能体（Agent）多做那么多事情。其实有一个家伙，他就说：“我不在乎可能大约八个月拿不到期权，我就是要加入。”Hamish 呃相当厉害。

英文原文

So yeah, we spun AMP out and we had to figure out a lot of new ways of working with these constraints that we had no idea that we were going to be under and we couldn't hire as I said. So as a result while our uh this is a log chart by the way while you know all the other big competitors in the space were growing and building old-fashioned companies in order to go and exploit this amazing technology of AI we had to stay at 21 people. But that was awesome because it was a forcing function for us to work differently and to figure out how can you do so much more with an agent where we we couldn't hire people. There's actually one guy who was like, I don't care that I can't get options for like eight months and I'm going to come join and Hamish is uh pretty awesome.

围绕 Agent 重构工作流与高频发布

所以我想谈谈我们构建 AMP（也就是我们的 Agent 产品）的方式，它是如何围绕这些限制迫使我们的工作方式而构建的，以及我们是如何围绕 Agent 成长起来的。在这里，我们摆脱了所有其他的工具。我们摆脱了所有你必须去查看的其他东西。显然没有 Jira，没有任务跟踪。我们没有客服支持系统。我们在 Agent、在 AMP 中所做的一切，都被称为 threads（线程）。我们甚至在 Agent 内部沟通。你可以说，嗯，你知道，“我发现产品里有个地方坏了。我们来修一下吧。”Agent 能倾听所有内容，然后去把它修好。

英文原文

So I want to talk about the ways in which we built AMP which is our agent product around the way that these constraints forced us to work and how we grew up around the agent. Here we got rid of all of the other tools. We got rid of all of the other things you got to check. Obviously not Jira, no task tracking. We don't have a support system. Everything we do in the agent in AMP, they're called threads. We even communicate in the agent. You can say um you know, I made a something's bad in the product. Let's do it. The agent can listen to everything and then go fix it.

如果你是在诸如笔记本电脑上运行的本地 Agent 里，当你意识到自己应付不来时，你是没办法拉别人进来的。这真的很有价值。我们推代码的频率高得过分。嗯，我想展示一下，你知道，我们实际上是如何在这里降低风险的。因为你可能会说——呃，让我、让我重新说起。你可能会说：“好吧，如果你一直在频繁推代码，那你怎么做到不把网站整垮呢？”区别就在于，当你可以推代码，而且一天中的任何时间你独自一人就能完成时。在着手进行一次大变更之前，你可以把工作切分成小得多的碎片。你可以去说：“让我添加日志记录，以确保我的理解、Agent 对系统的理解是正确的。让我推一小点改动，一小点改动，你知道，然后 Agent 可以对其监控一个小时，接着再推下一次改动。”你可以让一整套影子系统与旧系统并行运行，看看它是否在每一个单独的情形下都计算出完全相同的结果。

英文原文

If you're in like a a local agent that runs on your laptop, you can't bring someone else in if you realize you're in over your head. Really valuable. We push way too often. Um, and I want to show, you know, how we actually reduce the risk here. Because you might say, uh, let me let me start that again. You might say, well, if you're pushing constantly, then how do you not take down the site all the time? And the difference is when you can push and you can do it any time a day just by yourself. You can chunk up your work into way smaller pieces before you're going and doing a big change. You can go and say, "Let me add the logging to make sure that my understanding, the agents understanding the system is correct. Let me push a little change, a little change, you know, and then the agent can monitor it for an hour and then push the next change." You can run a whole shadow system side by side with the old system to see does it compute the exact same results in every single case.

那种工作方式实际上降低了风险，因为等你按过去的方式做完时，以如今事情必须推进的步伐以及客户的预期，你很可能早就忘了自己正在做的是什么了，那体验简直烂透了。嗯，让我再发几顶帽子。谁还想要帽子？好的，给你。抱歉，那位正在看笔记本电脑的兄弟。这是个教训，对吧？

英文原文

And that way of working actually reduces the risk because by the time you know you finish the old way of doing it and the pace that things have to move these days and what your customers expect, you probably forgotten what you're building and it just totally sucks. Um, let me give out some more hats. Who wants some more hats? All right, there we go. Sorry, the one guy who's on his laptop. That's a lesson, right?

给你。好的。

英文原文

There we go. All right.

视觉验证与批量生成

嗯，另一件事是让 Agent 能够无可辩驳地证明它正在构建的东西是正确的。嗯，它有很多种方法可以做到这一点。事实证明它们非常擅长证明这些事情。所以，如果你试图消除 UI 中的卡顿抖动，确保它加载时顺畅利落，你可以要求 Agent 去做一件人类绝不会有耐心去做的事情，那就是抓取每一帧并在视觉上进行对比。而这正是这里我所做的事情。

英文原文

Um, another thing is making it so the agent is able to irrefutably prove what uh the the thing it's building is correct. Um, there's a lot of ways it can do this. Turns out they're really good at proving these things. So, if you're trying to remove like jitter in the UI, make sure that when you load it's like nice and crisp, you can ask the agent to do something that no human would ever have the patience to do, which is take every single frame and compare them visually. And that's what I did here.

你可以要求 Agent 去为某样东西制作大量的不同版本。你永远不能要求一个人去——比如你知道，如果你让一个设计师去做这个东西的九次迭代或九种样式。他们会非常恼火。我清楚这一点，因为我和设计师合作过，我知道他们不喜欢那样。但对于 Agent 来说，是的，你知道，你可以让它制作大量不同的东西。这是 AMP 中语音听写的波形图。

英文原文

You can ask the agent to go make a ton of different versions of something. You can never ask a human to like, you know, if you ask a designer, go make nine iterations or nine styles of this. They're going to get really annoyed. I know because I've worked with designers and I know they don't like that. But for the agent, yeah, you know, you can have it make a ton of different things. This is the uh waveform for dictation in AMP.

这也是给那些说“我不喜欢干等我的 Agent”的人的解决方案之一。你知道，他们一次只要求一件事，然后说：“做这个小改动。”你不应该那样做。去让它一次性做大量的东西，你会搞定多得多的事情。

英文原文

And this is one of the solutions to people who say, "I don't like waiting on my agent." You know, they're asking for one thing at a time and then they say, "Make this little change. You shouldn't do that. go have it make a ton of things at once and you're gonna get a lot more done.

全闭环 Bug 修复与用户回馈

我们有这样一个神奇的系统：如果你在 AMP 中报告一个 Bug——这里有人在 AMP 中报过 Bug 吗？好的，太酷了。呃，对于那个 Bug 我深表歉意，但我们有一个全闭环的系统，当您报告 Bug 时，我们就会修复它。当然，你知道，Android 应用，我们显然会用 Agent 来修复它。我认为现在每个人都这么做了。我是说几乎每个人。很多时候工程师和客服团队依然是分开的，这有点不可思议。嗯，我们会给那个人发一封电子邮件表示感谢。我们会给他们免费的点数额度，这总是挺不错的。我认为很糟糕——我认为当产品中出现 Bug 而公司不道歉时，那很糟糕。我只是觉得这在道德上是说不过去的。接着我们会做的是，我们会去查日志。而这是人类单凭自己绝不可能去做的事情。我们会找出受该问题影响的所有其他人，因为无风不起浪。所以，你找到了遇到同样罕见错误的另外 17 个人。他们大概还记得这件事，而你主动告诉他们：“你多少次收到过某个产品的邮件说：‘嘿，我知道你昨天遇到了一个错误，那真的挺烦人的。为此深感抱歉。这是 10 美元。我们已经修复了。’？”顺便提一下，这是一个自我强化的良性循环，收到邮件的人知道你在意，未来他们会提交更多反馈。然后你把这些全发布出去，然后，你知道，搞定。而人们常说 vibecoded（全凭感觉用 AI 写的）产品都是垃圾，这就是那种循环。AMP 基本上 100% 都是用 AI 构建的，大家说它质量非常高。这就是你可以借助 Agent 以不同方式完成的事情之一，但如果每次都必须通过 PM，或者去找负责这个 Bug 的团队以及设计师，你无论如何也绝不可能做到这一点。

英文原文

We have this magical system where if you report a bug in AMP, has anyone here reported a bug in AMP? All right, cool. Uh I apologize for the bug, but we have a system that is a full loop where when you report a bug, we will fix it. Of course, you know, Android app, we'll fix it obviously with the agent. I think everyone does that now. I'm mostly everyone. A lot of times still the engineers are different from the support team, which is kind of crazy. Um, we'll send an email to the person thanking them. We give them free credits, which is always kind of nice. I think it sucks. I think it sucks when there's a bug in a product and the company doesn't apologize. I just think that's like morally wrong. And then what we'll do is we'll go through the logs. And this is something that no human would ever do by themselves. We'll see everyone else who was affected by that issue because where there's smoke, there's fire. So, you find the 17 other people that hit that same obscure error. They remember it probably and you proactively tell them, "How many times have you gotten an email from some product that says, "Hey, I know that you saw an error yesterday and that was really annoying. I'm really sorry about that. Here's $10. We fixed it." And by the way, that it's a self-perpetuating cycle where then people who get that know that you care and they're going to send more feedback in the future. And then you ship those all out and then you know, boom. And that's the cycle that people talk about vibecoded products being junk. AMP is basically 100% built with AI and people say it's very high quality. And this is one of the things that you can do differently with an agent, but there's no way in hell you could do this if you had to go through a PM or go to the team that owns this bug and the designers each time.

语音听写与极简工具链

嗯，语音听写。谁对他们的 Agent 使用语音听写？是的。哇。那可、那相当不错。那比我想象的要多得多。这是将信息位从你的大脑传递到 Agent 最快的方式。而且呃我认为这其中可能存在某种联系，呃它将迎来远程办公的回归，因为今天我正好在办公室里和嗯 Bang（另一位 AMP 联合创始人）在一起，一边做语音听写，那感觉呃非常恼人。嗯听写，如果你现在还没用起来，你一定要去用。只要克服那种尴尬感就行。呃你知道，呃像内森·菲尔德（Nathan Fielder）附体一样，告诉自己，你在说话时可以承受任何程度的尴尬。

英文原文

Um, dictation. Who dictates to their agents? Yeah. Wow. That's ah, that's pretty good. That's a lot more than I thought. It is the fastest way to get bits of information from your brain to the agent. And uh I think that there's probably something around it uh ushering in the return of remote work because I was just in an office today with um Bang, another AMP co-founder, dictating and it was uh very annoying. Um dictation, if you're not doing it yet, you got to do it. Just overcome the embarrassment. uh you know uh channel Nathan Fielder and tell yourself that you can withstand any amount of awkwardness as you talk.

我们已经将我们的 Agent 连接到了很多不同的工具。我想很多公司都已经走到了这一步，你知道，达到了这个阶段。我们已经在生产环境中这样做了很长时间。我们发现的是，我们越是开始使用这些直接由 Agent 使用的工具，我们甚至就越不再需要另一端的软件产品了。而在我们看来，实际对我们具有持久粘性的产品数量非常少。所以嗯你知道，我们在 AMP 中构建了一些新东西，例如一个 CRM。一年前你听到人们谈论说：“噢，没人会从零开始靠 AI vibe code 出一个 Salesforce 来”，是的，那完全是真的。你为什么会想要那么做呢？它——你用过它吗？如果你 vibe code 出了那玩意儿，而最终出来的就是那个成果，你会感到自豪吗？而且你只用到了 Salesforce 大概 2% 的功能。所以实际上，你知道，这——人们曾嘲笑这个想法，但实际上，你会不会去编写 Salesforce 功能的 2% 子集，然后再编写另外 2% 以最完美的方式与你其余的技术栈集成，因为 Agent 非常擅长做这个。而那，你知道，占 Salesforce 总体的 4%，那就像是轻松——而且比起 Salesforce，你更清楚你想要从中得到什么。并且你肯定不想花时间去和某个

英文原文

We've hooked up our agent to a lot of different tools. I think a lot of companies have gotten here, you know, to this point. We've been doing this in production for a long time. And what we found is the more that we start using these tools where just the agent is using it directly, we don't even need the software product on the other end. And it's a very small number of products that we think actually have sticking power for us. So um you know we built some new things in AMP for example a CRM and a year ago you heard people talking about oh no one's going to vibe code Salesforce from scratch and yeah that's totally true. Why would you want to? It's have you ever used it? You would you be proud if you vibe coded that and that came out the other end? And also you use like 2% of the features of Salesforce. So actually, you know, this people ridiculed this idea, but actually would you code the 2% subset of Salesforce features and then another 2% that integrate with the rest of your stack in the most perfect way because the agent is so good at that. And that, you know, 4% of Salesforce total, that's like the easy And you know what you want out of that better than Salesforce. And you certainly don't want to spend your time talking to some

Salesforce 的产品经理，听他们说：“我会去跟我的 PM 和高管团队争取，把它放进我们第四季度的路线图里。”诸如此类。因此，与那种情况相比，直接从头构建你想要的工具其实正变得越来越容易。这又回到了我们第一次尝试 Agent 时感受到的那种存在主义恐惧，但同时也是令人震撼的感觉。这就是我们之前料想会发生的事：它将彻底改变软件行业的一切。如果像 Salesforce 这样堪称 SaaS 界眼镜王蛇、拥有最可靠客户、最稳定续约率、几乎开创了这个品类的公司，如果我们都开始觉得“你看，我们为什么还要用他们呢？”，那么一切都开始崩塌了。如果你再也不能假设花 10 万美元雇一个销售去促成一笔交易，而且那笔交易能维持 10 年并且金额还会不断增长一点，整个 SaaS 模式就会开始瓦解。现在所有这些都成疑问了。

英文原文

Salesforce product manager and hearing from them. I'm gonna advocate with my PM and my executive team to get that on our Q4 roadmap. All that So, compared to that, it's actually getting easier and easier to just build the tools that you want from scratch. And this goes back to some of the existential kind of dread but also mindblowing feeling that we felt when we first tried an agent. This is what we kind of thought would happen that it would change everything about the software industry if Salesforce which is like the king cobra of SAS they have the most reliable customers most reliable renewals all that they like invented the category if we're starting to see you know why would we use them well everything is starting to fall and all of SAS starts to break down if you can no longer assume that you can pay a salesperson a 100K for closing a and that deal is going to last for 10 years and it's going to get bigger by a little bit. Not all of that is called into question now.

所以，如果你身在一家软件公司，那就是你们最终赚钱的方式，而你正试图敲响警钟、发出警告，那么绝对要拉响它，并跳出你所在的牢笼。因为在我看来，一旦你见识了那种规模小两个数量级的新型公司——在那里每个人什么都能做，每个人都身家利益休戚与共——那么你就什么别的都不想了。

英文原文

So if you're at a software company and that is how you ultimately make money and you're trying to ring the bell, you're trying to raise the alarm here, uh absolutely raise it and get out of the cage that you are in because uh in my opinion once you have seen how the new kind of companies that are two orders of magnitude smaller where everyone can do everything and where everyone has skin in the game, then you know you don't want anything

这就是我们所说的新型公司。我们拿掉了旧的组织架构图，现在我们 AMP 团队只有 21 个人。Brady 是在 Sourcegraph 做 SDR 的小伙子，非常聪明。有一天他说：“我要做一个电汇转账功能，这样如果金额太大刷不了信用卡，客户就能付款给我们。”他才大学毕业两年，就这样直接去做了，而在另一端根本没人知道。就像《纽约客》上那幅老漫画说的：在互联网上，没人知道你是一条狗。没人知道他是个 SDR，我们大家都在某种程度上既是开发人员又什么都做，但你确实能做到。而且你还能拥有对客户非常真诚的人。我认为有时客户需要听到大实话。比如我们团队的 Alex Keer，就在今天，他对一个想要我们产品的客户说：“嘿，听起来我们只会陷入一堆废话中，我认为这对谁的时间都不是个好的利用方式。”如果你能达到让大家都觉得自己有权说出这种话、真正为客户说出最合适的话的状态，那太棒了。你不可能从一个拿佣金的销售身上得到这种表现。

英文原文

And so this is what we call this new kind of company where you know we take that old or chart and now we just got 21 people on AMPs. Brady is this guy who is an SDR at source graph. Really smart guy. And one day he's like I'm going to make a wire a wire transfer thing so customers can pay us you know if it's too big for credit card and I'm and he's like two years out of college and he's just doing that and nobody knows on the other end. It's like that old New Yorker cartoon on the internet. Nobody knows you're a dog. You know, nobody knows that he's an SDR that we're all kind of, you know, devs and doing everything, but you can. And you also get people that are really real with customers. And I think sometimes customers need real talk. So, Alex Keer on our team, you know, just today, uh, he was like, "Hey, you know, to a customer who wanted our stuff, that sounds like we're just going to get bogged down in a bunch of BS, and I don't think that's a good use of anyone's time." If you can get to a place where you have people that feel empowered to say that and to truly say the best thing for the customer, that's great. You're not going to get that out of a salesperson who is commissioned.

我们还有一部分可能是某些人的最爱：我们每周只开一次会，除此以外的指导方针就是“跟随你的精力与灵感”。大多数公司你无法这样运转，但如果公司里的每个人都持有公司大量的股份——比如拿 100% 除以 21 等等——你实际上就可以信任大家。因为同样，当人们切身利益休戚与共时，这会让他们在所做的每一件事上都做得更好。这就像是一种激励，让他们站在全公司的角度思考，让他们做更多的事情，让他们更努力地工作。这就是为什么我也把周六和周日包含在里面的原因。

英文原文

We have this some people's favorite part. Uh we have one meeting per week and then everything else the guidance is follow your energy. um you couldn't run most companies like that, but if everyone at the company has a massive stake in the company, you know, take 100% divide it by 21, you know, and all that, you can actually trust people because again, when people have skin in the game, it makes them better at everything they do. It's like an incentive to make them think whole company, to make them do more, to make them work harder. And that's why I included uh Saturday and Sunday as well on this.

而且我们的发版交付量要多得多。我认为对许多人来说，AMP 是一款他们喜爱的产品。我们也有出色的业务，我们正在证明：这种盈利的、ARR 超过 1 亿美元的、按投入产出比算比传统老式公司效率高 40 到 50 倍的新型公司，是可以行得通的。

英文原文

And we are shipping uh just, you know, so much more. Um, I think that, you know, AMP for many people is a product that they love. We also have a great business and we're serving as proof that this new kind of company that is profitable that's more than 100 million ARR that is doing so on a dollar, dollar out, you know, basis that's 40 to 50 times more efficient than old-fashioned companies here that that that can work.

所以如果这对你有吸引力，如果你想推动你的公司朝这个方向发展，或者你想去创办一家这样的公司，欢迎联系我。我很乐意提供帮助。我的建议就是走出那个笼子。我听很多人说过：“我讨厌我们公司里那些八卦频道，大家都在那里议论变革。”你知道吗？你在八卦频道里待得越久，你在一家公司拿着高薪、非常聪明却无所作为的时间越长，你就开始成为问题的一部分了。当 Agent 已经彻底改变游戏规则时，你让自己继续仅仅做一名开发人员的时间越久，你就越是在伤害自己。所以如果这是你能带走的一点收获，那就是跳出那个笼子。让 Agent 真正塑造你的工作方式，并去一家愿意被 Agent 塑造的公司。

英文原文

And so if this is appealing to you, if you want to push your company in this direction, if you want to go start a company like this, then um reach out. I'm happy to help. And my guidance is just get out of the cage. I've heard from so many people who say, "I hate that there's a gossip channel with the people at my company. We're all talking about the change." You know what? The the longer that you are in the gossip channel, the longer that you are at a company getting paid a lot of money and you're really smart and you're not doing about it, you're starting to become part of the problem. And the longer that you let yourself be just a developer when agents have completely changed the game, you're really hurting yourself. So if this is, you know, the one thing that you take away, just get out of that cage. Let agents really shape how you work and get to a company that's willing to be shaped by the agents.

谢谢大家。如果你们想尝试 AMP，可以在上面获取并复制。我带了更多帽子，所以待会儿来找我。我可能之后会在 Chipotle 或别的地方吃晚饭。如果你想聊聊这些东西，欢迎来找我。好的，谢谢各位。[掌声]

英文原文

Thank you. And um if you want to try AMP, you can get up there. You can copy that. I've got more hats, so find me afterwards. and I'll probably be getting dinner at Chipotle or something. So, come find me if you want to uh chat about all this stuff. All right. Thank you all. [applause]

十二要素软件工厂与消灭劣质代码

随着 AI Agent 重塑软件构建方式，下一代软件工厂会是什么样？为了探讨 Agent 编程背后的设计模式与原则，请欢迎 HumanLayer 的 CEO 兼创始人 Dex Horthy。

英文原文

As AI agents reshape how software gets built, what does the next generation of software factories look like? To explore the design patterns and principles behind Agentic Coding, please welcome Dex Horthy, CEO and founder of Human Layer.

大家好啊！大家都怎么样？

英文原文

What's up everybody? How we all doing?

为现场所有的组织者欢呼喝彩吧。他们干得太棒了。Swixon 团队，让我们为他们鼓掌。还有所有的演讲嘉宾，继续为演讲者们鼓掌。[掌声]

英文原文

>> Give it up for all the organizers out here. They are killing it. Swixon team, let's get a round of applause. All the speakers, keep it going for the speakers. [applause]

我知道 Quinn 非常聪明，不过那场演讲真的很酷。我在后台看得很过瘾。大家好，我是 Dex。这是“十二要素软件工厂”，它最初只是个玩笑标题，但后来大家都这么叫了。还有“向劣质代码（slop）开战”。谢谢大家今天能来。我喜欢在网上聊编程 Agent，大约一年前我们做了整个上下文工程（context engineering）的内容，在 YouTube 上有很多播放量。直到大约一周前，我还是 HumanLayer 的 CEO 兼联合创始人，它是一个多人协作编程 Agent 工作区。对于那些庆幸自己不像我们有些人那样整天泡在网上的人来说，我们现在已经加入了 Factory，对此我感到非常兴奋，在这场演讲中我也会提到这一点。不过我想直接切入正题。去年夏天全都是关于这些——三个月前感觉就像是很久以前的事了——软件工厂全都是最热门的话题。我在六月份做了一次基于直觉的演讲，大概是讲我们可能仍然应该阅读代码，以及强化学习（RL）和验证器（verifiers）是如何工作的；就像我们谷歌的朋友所说的那样，模型确实很喜欢走捷径仅仅解决测试、仅仅让测试通过。现在我掌握了一些超出直觉的数据。有一个名为 slop code bench 的基准测试，实际上我们今天刚发布了一批新数据，来自威斯康星大学的实验室。这是一个非常未饱和的基准测试，GPT-5.5 得了 15%，Astra 我记得得了 16.3%，所以我们在这方面正在变好，但还有很长的路要走。我稍后会详细讲这些数据。

英文原文

Uh, I knew Quinn was really smart, but that was a really dope talk. I enjoyed watching it from backstage. Um, hi everybody. Uh, I'm Dex. This is 12 factor factories uh which started as a joke title but then everyone started calling it that. So uh and the war on slop. Uh so thank you all for coming. Um I like to yap about coding agents on the internet and like a year ago we did the whole context engineering thing. Uh a lot of views on YouTube and up until about a week ago uh I was the CEO and co-founder at a company called Human Layer. Uh the multiplayer coding agent workspace. Uh and for those of you who are blessed to not be uh quite as uh terminally online as uh some of us may be uh we are now part of factory which I'm very excited about and I will be touching on during this talk. Uh but I want to get right into it. Uh last summer was all about this like you know three months ago feels like forever factories were all the hype. Um, and I gave an intuition based talk back in June of like we should probably still read the code and here's how RL and verifiers are working and like exactly as um, our friends at Google were saying, you know, models really like just solving the tests, just getting them to pass. Um, I now have some data beyond just that intuition. Um, there's a benchmark called slop code bench which we just released actually today uh, a bunch of new data on uh, from the lab at University of Wisconsin. Um but very unsaturated benchmark you know GPT 5.5 got a 15% Astra I think scored 16.3 so like we are getting better at this but there's a lot of a lot of ways to go I'll talk all about that data in a sec

当然，我们还有来自 Faros 的研究：30% 的 PR 在未经审查的情况下被合并，评论更长、评论更多，每个 PR 的突发故障更多，每个开发人员引入的 Bug 更多。我给这个演讲起的名字，我觉得可能不太准确，我说的是“为什么软件工厂会失败”，然后大家都说“哦，Dex 是个反软件工厂的家伙，Dex 讨厌软件工厂”。我其实非常喜欢软件工厂，我是它的超级粉丝。那个标题可能更贴近于：你在软件工厂里可能会做某些事情导致它失败，这就是我认为可能包括的那些做法。而且，请千万不要那样做。也许就像那个暗黑工厂（dark factory）之类的东西。但我认为这里最核心的收获——而且我觉得我们已经从很多人那里听到了这一点——那就是软件工厂其实就是扎实的优秀工程实践。播客的好朋友、大会的好朋友 Lauren 昨天还在谈论这个，很多东西就是我们长期以来一直在做的事情。实际上，Factory 的 Eno 曾发过一段话，原文要长得多，但意思是：工厂思维关乎我们如何规范化流程，而不仅仅是规范化产出。而且还有整套

英文原文

we of course have the study from Ferros of you know 30% of PRs merging without review longer comments more comments more incidents per PR more bugs per developer uh and I I titled this talk I don't I don't think it was quite correct I said why software factories fail and everyone said oh Dex is the anti-software factory guy Dex hates software factories. I actually love software factories. I'm a huge fan. Uh the the title was probably more along the lines of, you know, there are certain things you could do in your software factory that would cause it to fail. And uh here's what those things what I think those things might be. And you know, please uh please don't do them. Uh maybe the dark factory thing. Um but I think the main takeaway here, and I think we've heard this from a lot of folks, uh is software factories is just good engineering. um friend of the friend of the friend of the pod, friend of the conference. Uh Lauren was talking about this yesterday of like a lot of this is just stuff we've been doing for a long time. Actually, Eno from factory had a really the post is much longer, but it's like the factory mindset is about how do we standardize our process, not just the output. Um and there's a whole

软件工厂与前向/后向压力

关于我们对这方面的思考，有许多内容可以深入探讨。但我试图说明的是同一个观点，对吧？标准软件工厂的运作模式通常是：有几位工程师，一些产品经理，可能还有一位 CEO 或某个人来掌舵愿景；这些人会提出我们想要构建的功能，并录入追踪系统。这依然是 AI 时代之前的模式。我们今天听到的许多关于软件工厂的内容，其实已经存在了 30 年，对吧？你从追踪系统里领取任务，构建它，运行单元测试、自动化测试，也许在本地调试一下。准备就绪后，提交一个拉取请求（Pull Request）。此时，可能会有人拉取代码下来交互测试。你的 CI 会运行一系列检查和静态代码检查工具（linters），这些其实都不是什么新鲜事。如果发现任何问题，就退回修改，修复问题；当它真正准备好发布到生产环境时，我们会做一些金丝雀发布、灰度推送、特性开关（feature flags）以及各种有趣的操作。最终，它接触到了我们的用户。我很喜欢我们的用户，很喜欢客户。但是，他们确实喜欢抱怨哪里坏了，要求我们还没开发的功能。于是这些反馈又回到了团队。有时在凌晨三点出了故障，客户往往是最先发现的。所以我们加上了监控，这样就能在凌晨三点把人叫醒，让他们在其他人察觉之前把问题解决掉。

英文原文

bunch of stuff we'll dig into there on like how we're thinking about that. Um but I try to kind of make the same point, right? is like your standard software factory is going to look something like you have some engineers, some product managers, maybe a CEO or someone to drive the vision and uh those people are going to come up with stuff that we want to build and they're going to put it in a tracker. This is again this is pre-ai. A lot of the things we hear about software factories today were around for 30 years, right? You take stuff off the tracker, you build it, you run some unit tests, automated tests, maybe you poke it locally. When you're ready, you make a pull request. At that point, someone might pull it down. Uh interact with it, test it. Um your CI is going to run a bunch of checks and static llinters and like none of this is like that new. Uh and if anything is wrong with it, you send it back, you go fix the problems and uh when it's actually ready to go to prod, we do some canaries and rollouts and feature flags and all kinds of fun stuff there. Uh and eventually it makes contact with our users. And uh I love our users. I love customers. Uh but uh they do like to complain about things that are broken and uh ask for stuff that we haven't built yet. Uh and so that goes back to the team. Uh and at some point something breaks at 3 in the morning and the customers are the first one to figure it out. So we add monitoring so that we can wake people up at 3 in the morning and they can solve it before anybody notices.

这就是你所谓的 AI 时代之前的软件工厂。基本观点在于：要让它变成 Agent 化，你所需要做的仅仅是用“Agent 来构建”替换掉“人工构建”。于是你有了编排能力。我这里不打算深入探讨沙箱、模型以及所有这些细节。但是，你可以通过引入我称之为“前向压力”（forward pressure）的东西来逐步构建它。是什么促使工作启动？我们能不能把所有功能需求直接路由到我们的工厂？我们能不能把发生的所有线上故障也接进来，这样当我凌晨三点收到报警呼叫时，我看到的是一个我们认为能解决该问题的拉取请求，而不仅仅是一段说明出了什么错的调用堆栈追踪？

英文原文

Uh this is your preAI software factory and the basic point is like all you need to do to make this agentic is replace someone builds the thing with an agent builds the thing. Uh so you have orchestration. And I'm not going to go into the details of the sandbox and the model and all this stuff. Um, but you can incrementally build this up by adding what I'll call forward pressure. What causes work to start? Can we take all of the feature requests and just route them straight to our factory? Can we take all of the incidents that happen so that when I get paged at 3 in the morning, I'm looking at a pull request that we think is going to solve it, not just uh a stack trace about what went wrong.

这就是前向压力，即是什么促成了新工作的开启。而“后向压力”（back pressure）则是指：好的，太酷了，现在我们可以在几分钟或几小时内构建出东西；但如果我们仍用老旧的方式，代码审查和测试依然需要耗费数小时甚至数天。因此我们引入了 Agent 化的代码审查、Agent 化的测试。现场有谁正在用 Agent 来审查他们的拉取请求吗？

英文原文

Um, so the forward pressure, what causes new work to start? Uh, and back pressure, which is sort of like, okay, cool. Now we can build things in minutes or hours. Uh but the review and testing still takes hours or days if we're doing it in the old way. So we bring in agentic code review. Agentic testing. Anyone doing agent agents code reviewing their poll requests?

好的。我本以为会更多一些，不过这也挺不错的。你们大家都在跟上步伐了。然后，我们大力投入到 CI/CD、代码检查工具中，并对代码进行类似于 Lean 检查的形式化验证。太棒了，很硬核，我喜欢。

英文原文

Okay. I thought it would be a little bit more, but that's pretty good. Y'all y'all are catching up. Um and then we invest heavily into our CI/CD and our llinters and our doing like lean checks on their code like formal verification. All right. Hardcore. I like it.

构建软件工厂的永恒法则与代码劣化

所以，关于后向压力：我们该如何让 Agent 在无需人类介入的情况下，通过爬山算法迭代出更好的结果？我们如何从整体上提高构建步骤产出物的质量？Agent 在这里确实能帮上忙，但让软件工厂变得优秀的许多东西，其实都是我们几十年来就已经拥有的。就像“上下文工程”（context engineering）一样——那是我去年夏天（大概在 2025 年 6 月）讨论过的事情，当时充斥着 RAG、记忆、状态、历史记录等各种概念，而我们试图提炼出构建 Agent 所需的永恒不变的核心要素。因此，我现在想提炼出在构建和设计软件工厂时，我认为那些同样永恒不变的要素。因为现在充斥着大量的噪音和炒作。正如我所说的，网络重度成瘾（terminally online）——如果你花太多时间去琢磨别人都在做什么，Twitter 真的会把你的脑子搞乱。

英文原文

Um so the back pressure. How do how do we let agents hill climb to a better outcome without needing to involve humans? How do we just overall increase the quality of what is coming out of our build step? And agents help here, but a lot of what makes software factories good is stuff we've kind of had for decades. Uh, and just like context engineering, uh, which was a thing that I was talking about last summer in like June of 2025, there was all these ideas of like rag and memory and state and history and all this stuff. And we tried to like distill out what are the timeless things that go into building agents. Uh, and so I want to distill out now some of like what I think are the timeless things that go into building and designing a software factory. Um, because there's a lot of noise and hype. Uh, like I said, termally online, Twitter is going to break your brain if you, uh, spend too much time wondering what everyone else is doing.

我接下来还会聊聊垃圾内容（slop），因为我和 Swix 以及这里的几位讲者一直在做一个非常有趣的项目。不过我们稍后再谈那个。我们要讲软件工厂的 12 条绝佳实践。我想我最初开始谈论这个话题时只有大约 9 条，但现在我们增加到了 12 条。所以我对此感到非常兴奋。

英文原文

Um, I'm also going to talk about slop uh, because I have a very fun project that I've been working on with Swix and a couple handful of other speakers here. Uh, but we'll get to that later. Um, we're going to do 12 great practices for software factories. And I think the first time I started talking about this, it was only like nine, but we are at 12 now. So, I'm very excited about this.

我刚才触及了这一点：“循环”（loops）曾经是一个非常风靡的概念。但如果你从前向压力和后向压力的相互作用来思考，我认为这会有效得多。所以，你看，它再次被炒得火热，但实际在战术层面上是：你如何自动启动工作，以及如何确保最终输出交由人类处理时，所有显而易见的简单问题（lowhanging stuff）都已经被修复了？让产出结果在自主范围内做到尽可能好。

英文原文

Um, I touched on this, but loops was this concept that was very in vogue. I think this is much more useful if you think about the uh interplay of forward pressure and back pressure. Um, so, you know, again, very hyped up, but the actual tactical thing is like how do you start work automatically and how do you make sure that whatever comes out and is touched by a human by the end, all the lowhanging stuff is already fixed. The outcome is as good as it can be uh autonomously.

第二条我称之为“知己知彼”（know your enemy）。我依然坚持认为，缺乏监管的模型在随着时间推移维护和提升代码库质量方面表现并不好。因此，如果你在乎构建一个在三到六个月后仍然具有可维护性的系统——我之前做演讲时就说过，请务必去读代码，因为其中很多都是代码垃圾（slop）。Agent 在解决一次性问题、制作电子游戏、Blender 废料等方面变得非常厉害。但如果你指引模型去修复你的代码库，它是可以去修复，但前提是你必须清楚里面有什么以及你想要什么。因此，如果你完全甩手不管，只是把功能塞进机器里，你可能会发现它并不会随着时间推移自主让情况变好。

英文原文

Uh number two I will call know your enemy. I will still posit that uh unattended uh models don't do a great job of maintaining and improving codebased quality over time. So if you care about having a system that's going to be still maintainable in three or six months. Um I mean again I gave this talk I said please read the code because a lot of it is slop. Uh agents got really good at solving one-off problems and making video games and Blender slop and all this stuff. uh I don't think you know if you steer a model to fix your codebase it can go fix your codebase but then you have to know what is there and what you want uh so if you are just hands off pushing features into the machine you may not find that it autonomously will make things better over time

代码演进基准测试

那时我还没有基准测试，现在我有一个基准测试了，它基本上是在检查点中让模型去解决问题。比如，一个模型可以在几天内完整克隆出整个 Excel 或 Redis，现在借助 Agent 集群甚至可能只需要几个小时；但这需要预先知道整个问题。而我们大多数人并不是以这种方式编写软件的。我们大多数人都是先发布一个功能，然后再发布另一个功能，接着再发布一个功能。如果你在没有 AI 辅助的情况下手工构建，一路上你会意识到：“哦，那是个糟糕的抽象。我们需要把它集中化，或者我们需要把它移除。”所以这是一个衡量模型随着时间推移演进代码库能力的基准测试。

英文原文

um back then I didn't have a benchmark now I have a benchmark uh that basically has the model solve problems in checkpoints like a model can clone all of Excel or Reddus in a couple days uh maybe even a couple hours now with like agent swarms uh but that requires knowing the entire problem in advance. And most of us don't write software that way. Most of us ship a feature and then ship another feature and then ship another feature. And if you're building by hand without AI along the way, you say, "Oh, that's a bad abstraction. We need to centralize this or we need to remove this." And so this is a benchmark that metals measures a model's ability to evolve a codebase over time.

目前 Astra 再次拿到了 16%，非常不饱和。这里有大量的数据。我大概没有时间全部看一遍，但我们可以看一下代码质量指标，比如圈复杂度、每个符号的行数、高复杂度函数的百分比等。还有更多。有 200 条自定义规则，其中 50 条基于我们长期以来一直在做的事情，还有很多是基于观察模型编写的代码并将其与 AI 时代之前的代码仓库进行对比而定制的。我写了一篇长文探讨这个，我写了不少内容。运行这个基准测试实验室的 Gabe Warlansky，今天刚刚根据我们与他们合作进行的一些研究发布了这篇论文。这里面有很多内容，我大概没有时间逐一过一遍，但他们也测试了模型设计良好库和接口的能力。

英文原文

Um, and so Astra again is 16% very unsaturated. Um, there's a bunch of data here. I probably don't have time to look at all of it, but we can look at uh code quality metrics like uh cyclomatic complexity, lines per symbol, high the the percentage of functions that have high complexity. There's more. There's 200 custom rules. Uh 50 of them are based on things we've been doing for a while. A lot of them are custom based on looking at code that models write, comparing it to preAI repos. Um I wrote a long article about I've wrote a bunch. uh Gabe Warlansky who runs the lab that does this benchmark just uh put up this paper today u based on some of the research we did with them. Uh there's a bunch of stuff in here. I probably don't have time to go through all of it, but they also test the model's ability to design good libraries and interfaces.

其中最离谱的一点是：模型确实学会了在 Python 中使用异常而不是返回元组；有很多被认定为可能不太好的模式，在六个月前的模型中出现的频率，远低于在当今模型中出现的频率。所以这几乎让人感觉垃圾代码随着时间的推移正在被放大。而且 Claude 和 GLM 现在都在用 Python 2 的风格写 Python 3。所以我们看到 %s 格式化字符串又卷土重来了，而且它们默认不再使用 f-string 了。因此，这些都是需要注意的事情。你可能不在乎这个，但我在乎。对于 Fable 5.2，目前结论尚无定论。我很喜欢读 Anthropic 关于如何对新模型写 prompt 的博客文章。据称，如果你告诉 Fable 它拥有该代码库的所有权，它就应该保持代码整洁。我还没有测试过这一点。我们目前正在后台运行基准测试。不过对于我们这些不开发第一人称射击游戏（FPS）的人来说——顺便向 Ben 致敬，他做了一款很酷的游戏，你们应该去看看那个视频，太疯狂了——你应该知己知彼。你应该明白，带着这样的认知去工作：如果你需要你的代码库能维持很长……

英文原文

Uh the most the craziest one here is like models did learn uh to use exceptions instead of returning tupils in Python. uh there was a lot of patterns that were identified as like probably not great uh that were way less present in the you know six months ago models than they are in today's models. So it almost feels like we're like learn the the slop is amplifying over time. Um and Claude and GLM are now writing Python 3 in Python 2 style. So we have uh you know percent s uh format strings are back and they will not use fstrings by default anymore. So things to be aware of. You may not care about this. I care about this. Um the juryy's still out on Fable 5.2. I love reading the Enthropic blog post about like how to prompt the new models. And so the allegedly if you tell Fable it owns the codebase, it should keep it clean. I haven't tested this yet. We're running the benchmark in the background right now. Uh but for those of us not building FPS games, shouts out to Ben built a cool game. Uh you should go watch this video. It's insane. Uh you should know your enemy. You should understand that like work with this in mind that if you need your codebase to last a long

规划与构建权衡

时间，或者你的代码库已经存在了很长时间，呃，我们大概应该保持对系统和代码的理解。呃，为了做到这一点，我们喜欢在构建前先规划。呃，我极力推崇这个做法。你应该这么做。我们以前在旧软件工场里就是这么做的。人们意识到了这一点，特别是在构建耗费数小时或数天，以及测试和审查耗费数小时或数天的时候。即使这部分现在变快了，如果做得规范的话，那部分可能仍然很慢。我们会进行架构评审，会有 Sprint 规划，我们会作为团队进行沟通，并保持共识，希望能够降低后续返工的概率。我们希望在真正花时间去构建之前——花时间和 token 去构建，然后进入审查却不得不返工——相当确定我们是在以正确的方式构建正确的东西。

英文原文

time or your codebase has been around for a long time um we should probably maintain understanding of the system and the code. Um to do this we like to plan before we build. Uh I highly encourage this. You should do this. We used to do this before in this old software factory. People realize especially when building things took uh hours or days and when testing and reviewing things took hours or days. Even if this part is faster, this part is probably still slow. if you're doing it properly. We would have architecture review and we would have sprint planning and we would talk as a team and we would align in the hopes that we might decrease the percent chance that we'd have to rework something later. We wanted to be rather sure that we're building the right thing in the right way before we actually go and spend time building it time in tokens and then get to review and have to rework it.

呃，很多人呃也在这上面做过头了。我肯定也做过头过，你花六个小时写一份规范，然后觉得，我感觉直接硬上（yolo）也能搞定。呃，因为有时候你只用两句话的 prompt 就能直接硬上，如果东西比较小，你根本不在乎。大概有 50% 的几率你再追加几条 prompt，然后就搞定了。你可以花五个小时手写一份详尽的规范，也许之后有 10% 的内容你必须修改。呃，或者你也可以花 20 个小时纯手工写完所有代码，那样就差不多有 0% 的几率，好吧，有 0% 的几率你会因为智能体做的事而必须返工。要知道，人类也照样会犯错。

英文原文

Um, a lot of people uh overdo this as well. I've overdone this for sure where you spend six hours working on a spec and you're like, I feel like I could have just yoloed this. Uh because sometimes you can just yolo two sentence prompt and if it's smaller, you don't care. There's maybe a 50% chance you send a couple more prompts and then it's good. You could spend five hours handwriting a detailed spec and maybe there's a 10% of it you have to change afterwards. Uh or you could write all the code by hand in 20 hours and then there's like a 0% chance, well there's 0% chance you have to rework it because of something the agent did. You know, people still make mistakes, too.

呃，不过我是从预期痛苦的角度来考虑这件事的，比如这东西后续需要修改的可能性有多大，乘以修改起来有多难？呃，在规划阶段我不会跟模型争论按钮应该是什么颜色，但我确实想确保我们搞对了数据库模式之类的事情。呃，因为以后改起来真的非常困难。所以根据你的问题、你正在做的事情以及你的代码库，你大概应该决定自己想落在这个光谱的哪个位置。

英文原文

Um, but I think about this in terms of expected pain, like how likely will this have to change later times how hard will it be to change? Uh, I don't argue with the model during planning about what color the buttons going to be, but I do want to make sure we get like our database schemas right and stuff like that. Um, because it's really hard to change later. So based on your problem and what you're doing and your codebase, you should probably decide where on this spectrum you want to land.

可视化对齐与辅助测试

呃，在视觉上对齐。呃，我发布了一个叫 show me 的 skill，它针对的是不用去读大段文字。你能得到 HTML 原型模型，能得到表格，能得到图表，能得到所有这些针对人类大脑优化过的东西，你知道的，人类大脑是非常视觉化的。呃，我们已经，我想 KOD 说过，你知道，几百万年来我们一直在逃避捕食者、采摘水果。我们阅读文字大概才 5000 年。呃，不管怎么说，你可以获取这个，它大概有 40,000 次安装。

英文原文

Um, align visually. Uh I published a skill called show me which is like optimized for instead of reading walls of text. You get HTML mockups, you get tables, you get charts, you get all this stuff that you know is optimized for uh the human brain which is very visual. Uh we've been I think KOD said, you know, we've been running away from predators and eating fruit for uh for millions of years. We've only been reading text for about 5,000. Uh anyways, you can get this. It's got about 40,000 installs.

呃，你应该帮助智能体测试它们的工作。呃，屏幕截图、浏览器、视频、curl，任何你能用来帮助智能体的东西。再说一次，它会朝着结果去攀升。给它一个——如果你正在构建一个 iOS 应用，确保它能在运行的任何环境中构建、运行并测试该应用。

英文原文

Um you should help agents test their work. Um screenshots, browsers, videos, curl, whatever you can do to help the agent. again, he'll climb against an outcome. Give it an if you're building an iOS app, make sure it can build, run, and test that app in whatever environment it's running.

抵制低质代码（Slop）与工具生态

呃，你应该对抗 slop，在座有谁是用 TypeScript 写的吗？好吧，有些人犯懒了。我觉得应该不止这些人，但是〔笑声〕

英文原文

Um, you should fight slop with anyone here writing in Typescript. Okay, some people are feeling lazy. I feel like it should be more than that, but [laughter]

呃，这是来自 Dylan Moy 的代码库，他是 Cloudflare 的首席工程师，叫做 Anti-Slop。如果你还没用，如果你还没在用它，你应该去看看。它有大约一百条规则，旨在阻止智能体生成 slop。呃，Factory 也有一个 ESLint 插件，我大概应该提一下。呃，另外，你也可以把 slop codebench 里所有的 Python 代码质量规则 fork 出来。这些在你的语言里都是存在的。你应该用起来。

英文原文

um, this is a repo from Dylan Moy, who's a principal engineer at Cloudflare Anti-Slop. If you haven't, if you're not using this, you should go check it out. It's a hundredish rules on like stopping agents from building slop. Uh, Factory also has an ESLint plugin that I should probably mention. Um, and also again the you can go slop fork all of the uh Python code quality rules out of slop codebench. This this this exists for your language. You should use it.

呃，slop 是一种工具。并非所有的 slop 都是坏的。氛围编码（vibe coding）、原型开发、业余项目。如果你有个东西在 feature flag 后面，你只是需要弄清楚人们是否想要它，呃，在你投入时间之前。我认为这很棒。我不认为必须时时刻刻、每一行代码都做到完美。

英文原文

Um, slop is a tool. Not all slop is bad. Vibe coding, prototypes, side projects. If you have something behind a feature flag, you just need to like figure out if people even want it uh before you go put in the time. I think it's like it's great. I don't think there's like always always always every line of code has to be perfect.

呃，你应该给模型提供一个 feature flagging 系统，呃，PostHog、Statsig 或 LaunchDarkly 或无论什么，这样它们就能尝试新东西，看看效果是否更好，然后做出决策，呃，确定在你的工具里正确的内容应该是什么。

英文原文

Um, you should give models a feature flagging system, uh, post hog or stats or launch darkly or whatever it is, so they can try stuff and see if it performs better and then make decisions about, uh, what the right thing to be in your tool.

呃，你应该随时间不断复利。评测一切，从以往的会话中学习。

英文原文

Uh, you should compound over time. Eval everything, learn from past sessions.

统一工作台与软件工场

呃，还有你应该统一你的工作台（forge）。呃，我的意思是，你知道，我们过去常常在邮件列表中到处发送 git patch。我想 Linus 和 Linux 的那些人现在还在这么做，但大多数人把所有代码都放在 GitHub 上。呃，所以在 ANAF forge 之前，我们集中管理 patch，但是呃我们仍然有呃人们在 Slack 里把 Claude 的答案来回粘贴，希望呃不要搞太多轮，因为这就像是我从我的里面复制出来发给我的同事，他粘贴到他的智能体里，然后他们发回给我，然后我再粘贴到我的智能体里，也许我们能得到一个答案。所以把你的所有设计、决策以及你在 Notion、Google Docs 或无论什么里面的内容集中起来，呃，应该尽可能把这些东西拉得更近，这样你不仅能理解关于某个东西是如何构建的代码，还能理解注入其中的所有决策。

英文原文

Uh, and you should unify your forge. Uh, what I mean by this is, you know, we used to send git patches around on mailing lists. I think Lionus and the Linux people still do this, but most people keep all their code in GitHub. Um, and so before the ANAF forge, we have centralized patches, but uh we still have uh people pasting like clawed answers back and forth in Slack and hopefully like not doing too many rounds of that because it's like I copy out of mine and send it to my coworker, he paste it into his agent and then they send it back to me and then I paste it into my agent, maybe we get an answer. So like centralizing all of your designs and decisions and like whatever you have in like notion and Google docs and whatever it is uh should like you should try to bring those things as close as possible together so you can understand not just the code about what something was built but all the decisions that went into it.

呃，你不可能什么都买现成的。我认为在这个领域有些东西就是无法产品化。显然你可以购买虚拟机，或者你可以自带开发环境，我认为这对大多数团队来说是非常私有化的。呃，harness 当然你可以去买一个前沿的 harness，或者你可以使用像 open code 或 pi 这样极简的东西，然后对其进行深度定制。呃，然后你就会看到这个控制平面的全貌，比如你如何在会话和计划上进行协作？你如何进行自动化和调度？你如何响应事件？所有这些治理相关的东西。呃，我很兴奋能和 Factory 一起在这上面合作的部分原因，就是他们对这幅全貌有着宏大得多的视野，并且在这里解决掉了多得多的东西。呃，但是有些部分，特别是在最顶部的这部分，就像我认为没人能卖给你针对你代码的验证，因为只有你自己清楚；如果你拥有数百名工程师和数百个代码库，你有那些主见，而且你知道如何测试你的代码，呃，这不仅仅是一个你能买到的黑盒；有些构建块是无差别的，但软件工场的理念我认为是一种思考我们工作的方式。

英文原文

Um you can't buy everything. I think there's things in this space that just cannot be productized. Obviously you can buy VMs or you can bring your own dev environment I think is very personal to most teams. Uh the harness of course you can go get a frontier harness or you can use something minimal like open code or pi and customize the heck out of it. Uh and then you have this control plane picture of like how do you collaborate on sessions and plans? How do you do automations and scheduling? How do you respond to events? All this governance stuff. Uh part of the reason I'm really excited to be working with factory on all this stuff is like they have a much bigger view of this picture and have a lot more stuff solved here. uh but there are parts especially at the top here is like I don't think anyone can sell you verification of your code because only you know if you have hundreds of engineers and hundreds of repos you have those opinions and you know how to test your code uh not just a black box you can buy there are building blocks that are undifferiated but the idea of software factory I think is it's a way of thinking about our work

反 Slop 宣言

呃，所以这就是软件工场。我还想谈谈 slop。呃，工场的核心在于我们如何提高人类的杠杆率，对吧，我们如何从我们的时间中获得最佳回报。呃，我也谈了一点为什么人类应该参与其中，以及对我们正在构建的东西和正在做的事情做一个尽职的掌舵人意味着什么。呃，但我认为它不仅限于软件本身。所以，我非常激动地宣布，呃，反 Slop 宣言（Anti-Slop Manifesto）。

英文原文

um so that's software factories I also want to talk about slop um it's factories are about how do we increase human leverage right how do we get the best return on our time. Uh, and I've talked a little bit about why humans should be involved and like what it means to be a diligent steward of what we're building and what we're doing. Um, but it's I think it's bigger than just software. So, I'm very excited to announce the uh anti-slop manifesto.

呃，你可以去，如果你访问 slotmanifesto.com，你就能读到它。我稍后会把网址放出来。呃，但我们是与这里的很多人合作构建了这个宣言，包括与会者和演讲嘉宾。呃，特别鸣谢 Swix 和 Phillip。呃，它的签署人来自呃 ZoomInfo 和呃我想还有 xAI 以及 OpenAI，这里面有很多很酷的人。我鼓励你们去阅读并了解它，呃，成为其中的一部分，你知道的，但我会告诉你们里面写了什么。

英文原文

Um, which you can go if you go to slotmanifesto.com, you can read it. I'll put the URL up later. Uh, but we built this in collaboration with a lot of people here, both attendees and speakers. Uh, special shouts out to Swix and Phillip. Uh it has signers from uh Zoom Info and uh I think XAI and OpenAI and like there's a lot of cool people on there. I encourage you to go read it and check it out uh and be a part of you know but I'll tell you what's in it.

呃，我们重视结果胜过产出。我们重视理解胜过速度。我们重视自己的思考胜过模型给出的任何初稿。呃，我们重视更出色的工作胜过更轻松的工作。呃，我们重视改进系统本身胜过仅仅修补输出。

英文原文

Um we value outcomes over outputs. We value understanding over speed. We value our own thinking over whatever the model came up with as a first draft. Um we value better work over easier work. Um, we value improving the system over just patching outputs.

呃，我们的信念是，AI 是我们大多数人合作过的最有用的工具。但它能创造，就像一群智能体在一单天内生产的工作量，比一个人类在整个职业生涯中能审查的还要多。看看 OpenAI 搞出来的那些数学方面的东西。呃，所谓 slop，就是我们在生产某样东西时投入的精力，少于我们要求另一端的消费者为了理解它所花费的精力。

英文原文

Um, and the things we believe are that AI is the most useful tool most of us have ever worked with. But it can create a like a swarm of agents can produce more work in a single day than a human can review in their entire career. Look at some of the math stuff coming out of OpenAI. Um, slop is when we put less effort into producing something than we ask their consumer on the other end to spend understanding it.

呃，使用 AI 并不必然使东西变成 slop。呃，但当我们转手交出 AI 的输出，却不对其质量或读者的体验承担责任时，slop 就产生了。呃，我坚信——这我已经说了一年了——AI 无法取代人类的判断力。它只能放大我们的判断力，或者呃在某些情况下，也许放大了我们判断力的缺失。呃，而且 slop 是不对称的。呃，生产它的成本

英文原文

Um, and using AI doesn't necessarily make something slop. Um, but it happens when we pass off AI output without taking responsibility for its quality or for the reader's experience. Um, I believe strongly been saying this for a year now. AI cannot replace human judgment. It can only amplify our judgment or uh maybe perhaps our lack of judgment in certain cases. Um, and the slop is asymmetric. um the cost of producing

生成千行 PR 或十页文档的成本正逼近于零，但评审它们的成本仍相当稳定。当我们反复绕过问题而不是妥善解决问题时，劣质内容（slop）就会泛滥。它源于那些只知做加法而不倾向于做减法的工具。我不打算指名道姓，但我认为很多 slop 源于压力。各家公司都在推促员工去重新定义何为人力所能及。而且我认为 slop 会产生滚雪球式的累积效应。因此，知识库、聊天记录和代码中存在的 slop 越多，下一个智能体读取并复制的就越多。而在我们团队中流窜的 slop 越多，我们对抗它的意志就丧失得越厉害。随着时间的推移，一切都会恶化：沟通、质量以及诸如此类的所有事情。因此，作为签署……我们《反 Slop 宣言》的签署者……嘿大伙儿，这里是不是有点热？我可能得脱掉夹克了。这就行了。他们还没法给我一件印有 Factory 标志的衣服，但我相信很快就会有的。[笑声]

英文原文

a th a thousand line PR or a 10-page doc is approaching zero, but the cost of reviewing it is still uh fairly constant and slop proliferates when we repeatedly work around problems instead of solving them correctly. It comes from tools that love to add and are not biased towards subtracting. Um I'm not going to name names, but I I think a lot of slop comes from pressure. Uh companies are pushing their people to redefine what is humanly possible. Um, and I think slop compounds. So the more slop sits in our knowledge base, in chat, in code, the more of it the next agent reads and copies. Um, and the more slop flying around our team, the more we lose the will to fight it. And everything gets worse over time. Communication and quality and all of this. So, uh, as the sign we the undersigned of the anti-slot manifesto. H, guys, is it, uh, it's a little hot in here. Uh, I might just have to pop off the jack. There we go. Uh they couldn't give me one of these with a factory logo on it yet, but I'm sure it's coming soon. [laughter] Um

因此，我们这些反 slop 的签署者决定：人的时间和注意力极其宝贵。所以我们致力于明智地使用它。我们抵制 slop，因为 AI 应当让我们的工作更出色，而不仅仅是更轻松。我们针对高效引出人类意图以及高效的人类理解，来优化我们的系统、工具与产品。而且我们在与模型协同工作时保持独立思考。我们探究资源、代码和现有成果。在真正着手工作之前，我们切实希望理解问题。而且我们对所交付内容的意图、品味和后果承担责任。无论 AI 提供了多大帮助来制作它，我们都对其全权负责。当我们要把智能体撰写的内容发送给他人时，我们会说明这一点，并确保它确实足够优质，然后再发送到 Slack、电子邮件或任何地方。当有人发给我们 slop 时，我们同样必须予以抵制。我们必须友善而有同理心地告知对方，然后请他们先自行审查一下。对我们大多数人而言，这是一个必经的过程。这两方面我都经历过。因此我也不得不面对那种提醒。

英文原文

thus we uh the anti-slop uh undersigners resolve that human time and attention is incredibly valuable. Uh so we optimize for using it wisely. Uh we fight slop because AI should make our work better, not not uh not just easier. Um we optimize our systems and tools and products for the efficient elicitation of human intent and for efficient human understanding. Um, and we do our own thinking when working with a model. Uh, we explore resources, code, prior art. We really want to understand the problem before we actually get to work. Um, and we own the intent and the taste and the consequences of what we ship. However much AI helps us to make it, we own it. And when an agent wrote something that we're sending somebody, we make that clear and we actually make sure it's good before we drop it in Slack or email or whatever it is. Um, and when someone sends us slop, this is also like we have to we have to push back. we have to tell people, you know, kindly, empathetically, then ask them to like review it first. Um, and most of us, it's a journey. I've been I've been on both sides of this one. Uh, so I've had to have that note. Um,

我相信审查永远存在超额收益（alpha）。随着模型变得更好，我们可能不再阅读大部分代码。我们可能仍会审查计划、设计或结果；但如果你什么都不审，如果你没有带来任何思考，那么你得到的内容就会和所有人得到的 AI 输出完全一样。

英文原文

I believe there will always be alpha in reviewing something. Um, as models get better, we may stop reading most of the code. Uh, we may still review a plan or a design or a result, but if you don't review anything, if you bring nothing to the table, no thinking, then you're just going to get the same AI output as everybody else.

审查智能体、代码检查工具（linters）以及校验器都非常有用，但它们并不能——不好意思——并不能包办一切。因此眼下，我们仍会审查输出内容的一个子集。至于审查哪个子集，当然取决于风险程度。然而，交付了什么、谁在用它以及它如何帮助他们，远比代码行数、合并的 PR 数量，或者更糟的是并行运行的智能体数量重要得多。如果这能带你达成成果，那固然很好。但未经阅读的输出并不一定全都是坏事。你看，一次成型的 prompt、氛围编码（vibe coding），只要用在合适的地方，都是高杠杆的举措。Slop 只有在破坏我们的最终成果时才算有害。如果我们希望做出好产品、构建易维护的系统，希望为人们节省时间和精力，若 slop 破坏了这些，那它就是有害的。

英文原文

um agent reviewers, llinters, verifiers, they're very useful, but they are not, excuse me, they are not sufficient. So, for now, we'll still review some subset of the outputs. Uh what subset of course depends on the stakes. Um but what shipped, who's using it, and how it's helping them matters far more than lines of code or PRs merge or god forbid the number of agents were running in parallel. Uh it's cool if it gets you to the outcome. Um but unreed output isn't always necessarily bad. You know, slop is a cool oneshot prompts, um, vibe coding, these are all like high leverage things that are in the right place. Slop is bad when it undermines our outcome. If we want to make a good product, a maintainable system, we want to save people time and energy. Uh, if slop undermines that, then it's bad. Uh,

最后我想说，我们谁都没能完全参透这件事。我们每个人都在重新摸索自己接受和不接受的底线，这里没有唯一的正确答案，你必须自我权衡校验。世界变化得太快了。我的目标是提供这些原则，以便我们能在未来许多年里发挥人类的重要作用，安全、负责任且为每个人创造价值地驾驭 AI。大家可以访问 slotmanifesto.com 阅读并签署这份宣言。简而言之，就是对自己所做的事情上点心。我会留下来听 Tariq 的演讲，然后……那张幻灯片上写的是什么？好的，之后我会在走廊里转悠。欢迎过来打个招呼。保持学习，保持用心。我很高兴能与大家一起继续构建。非常感谢大家。

[掌声]

英文原文

and I guess finally, like none of us has this figured out. Uh, we're each relearning what we will and won't accept, and there's no single right answer here. You got to calibrate yourself. Um, the world moves fast. Um, my goal here is these are principles that we can use to embrace our important human role in harnessing AI safely, responsibly, in a way that creates value for everyone, um, for many years to come. So, you can read and sign the manifesto at slotmanifesto.com. Um, TLDDR, uh, you know, give a care about what you're doing. Um, I'm going to stick around for Tariq's talk, but then, uh, what is that slide? All right, I'll be hanging in the hallway afterwards. Um, come say hi. Uh, keep learning, keep giving a Uh, I'm excited to keep building with y'all. Thank you so much. [applause]

探索人与 AI 协作的新路径

……致力于探索人类与 AI 智能体协同工作的新方式。我们的下一位演讲者正在重新思考我们与 AI 驱动工具交互的方式。女士们先生们，请欢迎独立工程师、前 Anthropic 团队成员 Thoric Shahippar。太棒了。

英文原文

to exploring new ways humans and AI agents work together. Our next speaker is rethinking how we interact with AI powered tools. Ladies and gentlemen, please welcome independent engineer and former anthropic team member Thoric Shahippar. Amazing.

好的，那个动画太酷了，但这也并不是《魔兽世界》。我都不知道自己在这里做什么了。不过大家好，我是 Thoric，很高兴见到大家。我想和大家聊聊自离开 Anthropic 以来我一直在思考的几件事。这是跌宕起伏的一年，我产生了许多反思。我的结论关乎我所谓的“苦涩教训甜蜜面”，以及我判断失误的那些方面。

英文原文

Okay, that was such a cool animation, but also this not World of Warcraft. I I'm not sure what I'm doing here. Um, but hey guys, uh, I'm Thoric. Nice to meet you. Uh, yeah. Um, I wanted to talk to you about a couple things I've been thinking about since I left Anthropic. Um, it's been a wild year and, uh, there are a bunch of reflections I've had. Um, and my conclusion is about, you know, what I'm calling the sweet side of the bitter lesson and the ways I've been wrong.

我认为今年我在很多事情上都猜对了。我认为我在模型能够编辑和生成视频这一点上是对的，我大约在去年 6 月就在讨论这事了。我认为在模型能借助 HTML 理解自身工作这一点上我是对的。而在模型能够帮助你理解并梳理未知盲区以辅助规划和学习方面，我也是对的。所以，这些都是我判断正确的事情，我也感到挺高兴。我认为很多人的工作中可能也融合了这类东西，但我也犯过很多错误。我认为这是 AI 领域的现象之一：大家往往会把自己判断失误的事情直接遗忘封存（memory hole），嘴上说着“哎呀，我其实一直都预料到了 AI 会非常厉害”。

英文原文

So, I I think I got a lot of things right this year. Uh I think um I was right about models being able to edit and generate videos. I was talking about this like sometime in like June last year. Uh I was right about models being able to use HTML to understand their work. And uh I was right about models being able to help you understand and figure out your unknown to help like plan and learn about things. So, these were all things I was like right about and I was happy. And I I think that like um a lot of you might be incorporating this sort of stuff in your work, but I've been wrong a lot too. And I I think this is one of those things in AI where you sort of just like memory hold the things that you're wrong about and you're like, "Oh, like I was just always expecting, you know, AI to be great." Um

以下是我判断失误的一些事情。首先，比如“你只需要 Bash”。我认为曾有过这么一个时期，智能体极其擅长使用 bash，Claude Code 也是围绕 bash 工具构建的，所以大家觉得“好吧，我们直接把一切都做成 CLI，以后就只会有 bash 和 CLI，而这会席卷全球”。事实上事情并不是这样发展的。比如我现在其实更偏好 MCP 而不是 CLI；而且现在还有了代码模式（code mode）、技能（skills）以及渐进式呈现（progressive disclosure）。智能体作用于现实世界的方式实在太多了，它们当时使用 bash，不过是因为初期以特定方式训练而留下的奇怪残余。

英文原文

but here are some of the things I've been wrong about. So, first, like Bash is all you need. I I think there was this like moment where the agents were really good at using bash and cloud code was built around the bash tool and so you know people are like well we'll just turn everything into a CLI like you know like there's just going to be bash and CLI and that's going to like take over the world. Um that's actually kind of not how it turned out. Like I actually prefer MCP to CLI now you know but now there's also code mode and skills and progressive disclosure. like there's so many other like ways that agents can you know act on the world right and the fact that they were using bash was just like this weird remnant of you know like them being trained in a certain way at the start

然后就是诸如“你只需要 React”。我想有一阵子看起来模型非常擅长写 React，因此如果你想打造一款具备 AI 能力的应用，你就应该搞明白怎么用 React，应该去用 React Native，应该直接把一切都放在 Web 上。这也同样被证明不是真的。我觉得现在模型可以用 HTML 写，可以用 React 写，可以用 Swift 写，也可以用 Figma 专有的 JSON 格式写，而且看起来都大同小异，对吧？所以我曾经担心模型都是用 React 训练出来的，我们就会永远只能写 React，React 几乎会成为最后一种语言。而这几乎肯定是不对的，现在很多人都在编写自己的专属语言。

英文原文

then uh there's like react is all you need I think uh for a moment it seemed like the models were really really good at writing react and so you know if you wanted to create an AI enabled app you should figure out how to use react like you should use react native you should you know like you should just make everything on the web. Um and uh that also has not been true. I I think like now you know models can write it in HTML, they can write in React, they can write in Swift, uh they can write in the Figma proprietary JSON format. Uh and they'll all look roughly the same, right? And so I was afraid that like you know the models have been trained on React and so we'd just be writing React forever and React would be like the last language almost, right? And that's almost certainly not true, right? I think a lot of people are writing their own languages now.

另外一件我曾经非常担忧的事，就是诸如扩散模型会不会蚕食艺术。大家会觉得我们对这些模型、或者对这些模型创作的艺术类型将毫无掌控力，而它却能直接做出最精美的东西。事实也完全并非如此。我觉得现在大语言模型可以通过代码、通过接管你的电脑、通过使用 Blender 或 Photoshop、Figma 或任何你喜欢的工具来创作艺术，对吧？因此，这些基本上都是我曾经有过的担忧，关于苦涩教训、外星智能（alien intelligence）、修格斯（shoggoths），对吧？我们用这些词汇来描述那种担忧，害怕模型会有其独特的行事方式，对我们而言既古怪又陌生，而我们只需要……

英文原文

Um, and uh, I think one of the other things I was really concerned about was like diffusion models will kind of like eat art. Like it would just be like, you know, we wouldn't have any control over these models and or the kind of art that these models are creating. Uh, but it would just like create the most beautiful stuff. And that hasn't been true as at all. Like it I think now LM can make art through code through controlling your computer and using you know Blender or uh Photoshop or Yeah. or Figma or whatever you'd like, right? And so um these were all sort of like fears I had about like I think the bitter lesson, you know, art alien intelligence, the shogunuts, right? like like these are all words we use I think to describe this kind of like fear that the models will just have this like way of doing things and it will be weird and alien to us and we just need to

接受它，你知道，就像你只需要顺从这种外来智能一样。但我觉得事实并非如此，你知道。所以，我经常回想的一个问题是：好的，我们怎样才能既拥抱 AGI（AGI pilled），又保持以人为本？关于这种演进，我们能学到什么？对于自己曾经认识有误的地方，我又能学到什么？

英文原文

accept it you know like you just have to like submit to this like foreign intelligence um but that's just like I don't think that's what happened you know um and so I think the like the question I come back to a lot is like okay like how can we be AGI pilled and human- centered and what can we learn about you know this progression what we like you know what can I learn about what I was wrong about

我认为这非常非常重要，因为软件工程师处于 AI 应用的最前沿。在软件行业发生的一切，同样会发生在知识工作和经济的其他领域。因此，我们如何应对和运用 AI，也将成为其他人采取的方式。我非常喜欢这里的几场演讲，讨论了诸如对抗低质输出（fighting flop）之类的话题。我认为我们是第一道防线，对吧？或者是去怀抱更大的雄心。

英文原文

right so I think this is really really important because software engineers are like the leading edge of like AI adoption right like any thing that is happening to software will happen to knowledge work and other parts of the economy and so I think how we deal and engage with uh with AI is um yeah how other people will as well and you know I really enjoyed the different talks here talking about things like yeah fighting flop I think you know we're the first defense uh for that, right? Or like being more ambitious.

我倾向于将 AI 同时视为一场工业革命和一次文艺复兴。我认为今年真正的重点是工业革命。我们经常使用“软件工厂”这个词，也就是把软件从一种非常不可靠、难以执行的手艺，转变成一种不再是制约瓶颈的东西。这种工业化绝对正在发生，这确实正在软件领域发生。

英文原文

Um what I'd like to think about it as AI is both at the same time an industrial revolution and a renaissance, right? And I think this year was really about the industrial revolution, right? Like we use the word software factories very often. Um and uh like just sort of like turning software from this like craft that was very unreliable and hard to execute on into something that like is not the limiting factor, right? Um and this sort of like industrialization it's absolutely happening. This is definitely something that is happening with with software.

但与此同时，也正在发生一种文艺复兴：人们渴望质量越来越高的软件，而你倾注其中的每一分创造力、实用性、精神与灵魂，都是有用的，并且具有经济价值。因此，你同时面对这两件事，要同时兼顾并在脑海中保持这两者非常困难，但我认为这非常真实，而且我们正在同时经历这两者。

英文原文

But at the same time there's also kind of a renaissance happening right where people want and uh higher and higher quality software and every marginal bit of like creativity utility like spirit like soul that you put into it is like is useful and like economically valuable right and so simultaneously you've got both of these things that are very hard to juggle and and sort of like hold in your head at the same time uh but I think is very true and uh Uh yeah, we're doing both of them.

锯齿状前沿与平滑内部

很多时候谈到 AI 能力时，我们都会提到锯齿状前沿（jagged frontier）。大约两年前，你可能只是让 AI 帮你写某个函数；或者大约一年半、一年前，是让它构建一个功能；接着是“嘿，做个待办事项应用，做一个简易的待办应用”，然后是“现在帮我克隆一个 Slack”，对吧？而现在则是“好的，从零构建一个马里奥赛车（Mario Kart），然后把它移植到 Wii 之类的平台上”。能力上的提升是显而易见的。每当查看评估和基准测试时，你都会看到这种能力上的锯齿状前沿。

英文原文

So often times when you talk about AI capabilities, we talk about like the jagged frontier, right? And so uh you know, you maybe like you go back two years and you're like asking AI to like write this function for you. Um or you know, like like a year and a half ago or a year ago, it's like build this feature. Then you're like, hey, build this like to-do app. Build like a kind of simple to-do app and then clone Slack for me now, right? and and and now it's like okay build Mario Kart from scratch you know and then then like port it to like the Wii or something. there's like the capability increases are the things that you see right where you know whenever you look at the evals and benchmarks and stuff you see this like just um this jagged frontier of like capabilities

但我还想谈谈“平滑内部”（smooth interior）。模型的方式——哎呀，这里是有动画在播放吗？所谓平滑内部，核心理念是随着模型变得越来越好……这里的动画展现的是锯齿状前沿，以及中间的这个圆圈。这个圆圈大致代表了模型足够聪明去完成的所有事情。比如编写 React 代码，或者使用 Bash。

英文原文

but I also like to talk about like the smooth interior where the the way whoops the way the models is there an animation playing here um the smooth interior is like The idea is that the models are um as they get better, the the animation here is like this jagged frontier and this circle in between. And so uh the circle sort of represents all of the things that the models are too are smart enough to do. And so like examples being like uh writing react code, right? Or like using bash.

你会看到，在前沿之下并不是锯齿状的。如果它能写 React，它就能写 Figma，就能写 JSON，能把所有这些能力整合在一起完成，模型非常擅长处理极其广泛的任务。我认为这一点也非常重要，对我而言甚至更具冲击力，因为我已经不再那么关心它们能否从零构建马里奥赛车了。我真正关心的是：我们正在构建的软件质量高吗？如你所说，是否配得上一次文艺复兴？

英文原文

And what you see is that like it's not jagged below the frontier, right? like if it can write react, it can write Figma, it can write JSON, it can do sort of like all of these capabilities altogether and there's like a breadth of tasks that the models are really really good at. And this is something that's also I think really like important and um for me even more impactful because I don't really care that they can build Mario Kart from scratch anymore, right? What I really care about is like, hey, is the software that we're building is it high quality? you say worthy of like a renaissance, you know what I mean?

所以，你可以按照你想要的方式去做。如果你在做原型设计，你可以要求：“能给我一套 HTML 吗？能给我 React 吗？能生成一个 Figma 文件吗？你能操作我的电脑并使用 After Effects 吗？你能制作一段视频吗？”有各种各样不同的方式来获得最终结果。我对这一点感到非常满意。我之前真的很担心模型只会偏爱做单一的一件事，而我们只能依附于那一件事。

英文原文

So, um yeah, like I think that like you know, you can do things the way you want. So, like if you're building mockup, you could be like, "Hey, can I get an HTML? Can I get a React? Can I get a Figma file? Can I computer use my computer and use After Effects? Can you make a video?" There are all these different ways of getting the like final result. And I've been really happy with this. I really was afraid that there would just be like one thing that the models love doing and we'd only be doing that.

推演到极致，就像我们经常讨论的那样，到目前为止我们讨论过的一切——关于如何构建这些测试套件框架（harnesses）、软件工厂以及为模型搭建脚手架；甚至包括有一段时间我们还会认为“你的工作是给模型做质检（QA），或者审查代码”，但显然情况并非如此。我认为随着模型越来越聪明，它会更多地呈现出这种“平滑内部”，它们就是能把事情做好。

英文原文

Um, and you know like taken to the limit kind of like you know we talk about this a lot but like you know everything we've talked about so far about like you know how you build like these harnesses and software factories and um these like the scaffolding for the models even things like you know for a while we were also like hey um the model you'll like your job is to QA the model you know or review the code and that's clearly not the case right I think as the models get smarter and smarter um it will become more of this like smooth interior, right? Where they're just doing things well, right?

你可以想象，完美的模型将不再需要真正去验证自身的工作或扫描安全漏洞。它直接把代码写出来，然后说：“我已经做完了。”你去检查时会发现：“你是对的，按照既定的规格说明（spec）和目标，你的实现完全正确。”我认为这其实非常令人兴奋。

英文原文

And like the perfect model, you can imagine won't need to like really verify its work or scan for security vulnerabilities. We'll just like write the code and be like, "Hey, I'm done now." Um and you will check and you'll like, "Oh, you're right." Like, you know, according to the spec and the goals that you that we did, you wrote you did it correctly. Um and uh I think that's actually very exciting.

我喜欢用来直观展现模型知识储备程度的一种方式是这项评测——我想是 Karpathy 发布的。基本上，你向模型——我想这些是 Claude 模型——输入经度和纬度，然后问它：“这个坐标点是陆地还是水域，对吧？抱歉，是陆地还是水域？”由此绘制出一张地球地图。这是 Opus 4.6，6 大概在那边的某个位置，相当不错的软件工程模型，展现出有点模糊的景象。

英文原文

Um one of one of the ways I like to sort of think about visualizing how how much the model knows of this eval I think aparthy um posted it but basically you ask any I think these are these are cloud models. You ask a model uh you give it a longitude latitude and you say like hey is this point uh earth or water right uh ground sorry land or water and it from that you plot a map of the earth right and yeah this is opus uh 4.6 six is somewhere over there, right? Quite a good software engineering model. Kind of like a blurry view.

现在你看 Opus 5.5，几乎堪称完美。它清楚这些坐标点的每一处，基本上了解整个世界。所以我觉得，除了那些关于锯齿状前沿的基准测试之外，这能让你直观感受到另一种维度。我觉得那个前沿对我们而言已经变得越来越不重要，更重要的是模型所具备的通识底蕴。令人兴奋的是，我们可以用智能来理解并塑造我们使用智能的方式。

英文原文

And now you look at Opus 5.5 and it's like almost perfect, right? Like it knows every piece of, you know, these like these coordinates. It knows like the entire world basically. And so this I think just gives you an idea besides like, you know, the benchmarks about like the the the jagged frontier. I feel like that's less and less important to us and more important is just like the general knowledge that the models have. And what's exciting I think about this is that we can use intelligence to understand and shape how we use intelligence, right?

从工业化到直觉编程的困境

我认为这将是下一年的主题。今年真正的重点是软件的工业化，以及建立这种可靠的软件制造方式。但这具体是什么样的呢？我最近一直在开发一款游戏，就是在 Twitter 上发帖提过的那款。它是一款 2D 伪三维等距视角的动作游戏，灵感来源于《降世神通》（Avatar）和《任天堂明星大乱斗》（Smash Brothers）等作品。

英文原文

And I think this that will be like the theme of the next year, right? Where this year I think was really about the industrialization of software and and creating this like reliable way to make software. Um, but what does that look like? So, I have been building a game. Um, it's this like thing I've been posting on Twitter about. Um, it's a 2D like isometric game. It's like it's kind of like a fighting game inspired by things like Avatar and Smash Brothers.

做这个挺好玩的，但做起来也相当令人沮丧。我真的很努力地想在这里当一名优秀的氛围程序员（vibe coder）。但有时候我根本不知道每个角色到底能做什么，因为我会说：“嘿，我想让这个角色扔一块石头。”然后我会觉得：“噢，太棒了。”但在扔石头时会发生各种各样的情况：比如这个角色还可以召唤一堵墙，那么石头会穿过那堵墙吗？飞行的速度是怎样的？蓄力和体积大小又该如何设计？面对所有这些疑问，我发现模型直接替我做了一大堆决定。

英文原文

Uh, and I have kind of like it's fun, but it's also quite frustrating to do. Like, I'm really trying to like, you know, like be like a great vibe coder here. Um, I don't really know what each character does sometimes because like I will say like, "Hey, I want this character to throw a rock." And then I'll be like, "Oh, yeah, great. But like there all these like different things that can happen when you're throwing a rock. Like this character can also summon a wall. So does the rock go through the wall or not?" And like how what speed or like what what about size and charge and all of these questions. Um, and I I find that like the model just makes a bunch of choices for me

然后我试玩了一下，心想：“哇，我完全不知道这个角色到底能做些什么。”我很容易分心，很难进入心流状态。我发出去一个提示词，然后就开始做别的事情，之后再切回来。老实说，正因为这些问题，我的进展反而变慢了。这确实让人沮丧。有段时间我想：天哪，看来跟 AI 协作就是这个样子的，未来大概就是这么回事了吧。但反思之后我觉得：天哪，我不认为这种方式是对的。我认为我们……

英文原文

and then I try it out and I'm like, "Wow, I have no idea what this character actually does. I get distracted pretty easily. I don't feel in the zone very much, you know? I I sort of send off a prompt and I start doing something else and I come back. Um, and I honestly make less progress because of these things, right? And so, uh, I think this is frustrating." And I think for a while I was like, man, I guess this is just like how you work with AI. This is just like what it's going to be like, you know? But I think reflecting I'm like, man, I I don't think that's right. I think that like, you know, we

让 AI 按照我们想要的方式工作，对吧？所以，我认为这是不可接受的。我不认为这能带来复兴，对吧？我不认为这能带来令人惊叹的软件。部分原因，回到 Dex 说的，伟大的作品正是关乎这个混乱的过程，对吧？你不知道自己究竟想要什么。它更多取决于你做出的选择，以及什么没有被纳入或什么没有被包含，而不是包含了什么。所以，我认为你有点必须成为一名专家。你必须真正懂行，才能做出伟大的东西，对吧？才能做出配得上复兴的东西，对吧？当我看到某种 vibe coding 时，我会觉得，伙计，我完全摸不着头脑。我就是不太理解，你懂吧？

英文原文

have this like AI can work in the way that we want, right? And so, yeah, I don't think it's acceptable. I don't think that this is what leads to renaissance, right? I don't think this is what leads to like incredible software. Um, and part of it, you know, uh, going back to what Dex said is like like great work is just about like this this messy process, right, of like you don't know like exactly what you want. It's more about the choices you made and what wasn't involved or wasn't what wasn't included versus like what was included. And so, um, yeah, I think there's like, uh, you you like kind of just have to be an expert. You have to just really know your stuff in order to make something great, right? In order to like make something worthy of a renaissance, right? And I just like when I look at something vibe coding, I'm like, dude, I have no idea. Like, I just don't really get it, you know? Um, and so,

代码库分层与自定义脚本语言

那么，我在做什么？我该怎么解决这个问题？我的做法之一——而且我还处于非常早期的阶段，我并不是说我已经全弄明白了——但我认为我的做法之一是将我的代码库分成两部分。我的代码库里有类似引擎的部分，然后我有一个文件夹，我想我叫它 spec，但我在想干脆叫它 human 之类的。那个文件夹里的代码是我要读的，你懂吧，它是经过 lint 处理的代码，而且针对 Claude 有所有这些关于那部分代码应该是什么样子的规则。它感觉应该像英语一样。它有点像一份规格说明（spec）。我会向大家展示它是什么样子的。但它的密度达到了只要我读这段代码，我就知道正在发生什么，我能编辑它，我能微调它。但更重要的是，我能对我的游戏到底是什么样子掌握一点基本事实（ground truth）。

英文原文

um, what what am I doing? How am I fixing this? Um, one of the ways I'm doing, and I'm still quite early on. I'm not saying that I've like figured it all out, but I think one of the ways I'm doing it is that I have two parts of my codebase. I've got like this like engine part of the codebase, and then I have a folder like I think I call it spec, but I'm thinking I'm just calling it human or something. And that folder is the code that I read, you know, and um it's like this code that's linted and like there's all these rules for Claude about like what that code looks like. It's supposed to feel like English. Um it's kind of like a spec. And I'll I'll show you what it looks like. Um but it's at the density where if I just read this code, I know what's happening and I can edit it, I can tweak it. Uh, but more importantly, I just like, you know, can have a bit of a ground truth of like what the what my game looks like. And,

所以我们来做个快速演示。所谓演示，我想我的意思是我只是要……哎呀，这是我的 AIE 演讲。好的，来了。好的，这就是我的游戏。我这里有这个 game 文件夹。然后我这里有这段代码。这是 agent 代码，对吧？这段代码里，我会觉得，伙计，我不知道摄像机之类的到底是怎么工作的。这里有一大堆我不懂的代码。但是接着在这段代码里，我能看到我所有的角色，以及他们的技能。所以我可以进入这里，然后说，好的，这是投石动作，对吧？在这个动作内部，有很多调节它的方式。它注释写得很好。它使用像米（meters）这样的单位，你懂我意思吧？而不是像 x 设为 0.5 之类的。它试图一路上创建许多不同的抽象。因此我能看到，有用的事情之一是，这个角色有三种投石方式，对吧？比如可以从地面抓取，可以从墙上抓取之类的。我能看到类似角色的真正逻辑，对吧？我并不真正了解底层引擎是如何工作的，或者这些在底层是如何实现的，但我基本上为我的游戏创建了一种近乎自定义的脚本语言。

英文原文

um, so let's do do like a quick demo here. Um, by demo, I guess I mean I'm just going to Whoops. This is my AIE talk. Okay, here we go. Okay, so this is my game. I've got in here this folder game. And then I've got here this code. This is the like agent code, right? Like this is the code where I'm like, dude, I I don't know how, you know, cameras work or whatever. I, you know, like there's a bunch of code here that I don't understand. But then in this code, I see have all my characters and I have their abilities. And so I can go in here and I can say like, okay, this is the rock move, right? Um, and inside of this move, there's a bunch of ways to tune it. Um, it's well commented. It uses things like meters, you know what I mean? instead of like x like.5 or whatever. Um it uh tries to like create a bunch of different abstractions along the way. And so I can see that like one of the things that you know is useful is that like there are three ways to throw for this character to throw a rock, right? Like it can uh grab it from the ground, it can grab it from the wall or something. Um, and I can see sort of like the um the true logic of the character, right? I I don't really know how the engine works or how, you know, like this is implemented underneath, but I've essentially created this like almost a custom scripting language for my game.

这意味着，当我在玩这款游戏时，我可以微调细微之处。我仍在开发中。比如我说我希望石头大得多。所以我可以说比如 1.5，但我使用的是一个真正的 IDE。如果我转到我的游戏里，我可以看到我有一块石头，它可以蓄力，变得非常大，然后我把它扔出去，对吧？但这非常有趣，因为我也开始更多地使用 Cursor，更多地使用 Tab 模型。你懂我意思吧？因为现在我可以表示，要是我想说在中间召唤一堵墙，将它排队为一个动作，然后 Cursor 就会在这里进行 Tab 自动补全。这有点像规格驱动开发（spec driven development），但这份规格基本上就是这种脚本语言。

英文原文

Um, and so that means like, you know, when I'm playing this game, um, I can tweak little things. You know, I'm still I'm still like working on it. So like I can say I want the rock to be a lot bigger. So I can say like um you know 1.5 but I'm using an actual IDE uh and so if I go here to my game uh you know I can see that like I have a rock it can charge it gets really big and I throw it right. Um but I'm like kind of it it it's really interesting because I'm also starting to use cursor more and I'm using the tab model more. You know what I mean? Because now now I can be like, oh hey, what if I just want to say like you know uh if you summon a wall in the middle like cue it up as an action and then cursor will sort of like you know do the the tab complete here. Um, and this is kind of like it's kind of like spec driven development, you know, but the spec is this like this this scripting language basically.

这是我最终编写这款游戏的方式吗？我不确定。在某些方面，当我需要做复杂的事情时，我仍然需要让 Claude 来做，对吧？我需要让 Claude 制作原语，但对我来说理解正在发生什么是真正容易的。我感觉掌控度高得多。在 Claude 工作的过程中，我也能进行审查并与它协作。这一直非常令人兴奋。因此我认为对我的启发是，我现在必须思考为我交付高质量软件的最佳方式，对吧？我不再那么受限于模型的能力以及它喜欢用什么编写代码之类。我只是不必去想……我需要思考的是我怎样才能交付优秀的软件。

英文原文

And uh, yeah, it's like is this the final way I'm going to write this game? I'm not sure. It's like, you know, in some ways uh, when I need to do something complicated, I still need to ask Claude to do it, right? And I need ask to claude to like make the primitives, but it's actually really easy for me to understand what's happening. And I feel like much more in control. And while like cloud is working, I can also sort of review it and work with it. Um and uh that's been really exciting. And so I think the takeaway for me has been that like um the takeaway for me has been that like I now have to think about the way the best way for me to deliver high quality software, right? And I'm not as constrained and limited by like the model's capabilities and like what it likes to write code in or something. I'm just not thinking about like I need to think about like how can I deliver great software.

迎接软件复兴

所以我想，如果我们真正内化了这一点，那么接下来的一年将迎来一场软件的复兴。今天在这次大会上，许多人都谈到了如何更有雄心，我不认为这些有什么特别新鲜的，但我只是认为有许多事情可以做，比如新语言，对吧？显然，我们想要像 Rust 这样非常高效的东西，但我们也希望事物可能具备模块化和热重载能力，我们希望它可能具备可读性，对吧？新的原语，比如可能存在新型数据库、新型服务器，比如新的操作系统交互。我一直在 vibe coding 一大堆个人软件，它们能与我的操作系统进行交互。新的交互范式是可能的。我认为很多古老的人机交互（HCI）研究现在重新审视起来非常有趣。新的美学，我认为我们使用和设计软件的方式可以发生很大改变。

英文原文

And so I think if we like really internalize that I think then this the next year will be about a renaissance of software. And you know I think many people have talked today and you know this conference about how to be more ambitious and I don't think anything any of this is quite new but I just think there's like many things to do like new languages right like obviously like you know uh we want things like Rust that are very efficient but we also want things to be uh potentially modular and like hot reloadable um we want it to be potentially readable right uh new primitives like there probably new types of databases new types of servers like um new operating system interactions. I've been sort of like vibe coding a bunch of personal software um that sort of messes with my operating system. Uh there are new interaction paradigms possible. I think um a lot of like old HCI research is now like very interesting to revisit. Um new aesthetics I think the way that we like use and design software uh can change quite a lot.

我认为这里有太多事情可以做了，我不认为任何……我认为每位演讲嘉宾都谈到了这一点。我没有太多要补充的了，但我认为我真正想让我们说的是，我们需要做的是决定工作的未来是什么样子，对吧？并决定我们理想的体验是什么，因为模型并不是试图强迫我们采用某种特定工作方式的外星智慧。它们差不多什么都能做，而且它们也将会去做。但我们的工作是带来这场复兴。谢谢大家，这就是我的演讲。［掌声］

英文原文

And um yeah, I I think that like there's so much to do here that I think I I don't think that any of this is everyone every speaker I think has talked about this and um I I don't have too much more to add, but I think that like what I really want us to say is that what we need to do is to decide what the future of work looks like, right? and decide what our ideal experience is because the models are not, you know, this alien intelligence that's like trying to force us into a specific way of working. Uh they can kind of do everything and and they will. Um but our job is to like bring about the renaissance. So thank you guys. That's my talk. [applause]

现在，在［抽鼻声］CodeSummit 2026 落下帷幕之前，有一件大事我们想和大家一起庆祝。

英文原文

And now before [snorts] CodeSummit 2026 comes to a conclusion, there's something big we'd like to celebrate with all of you.

大会总结与现场互动

大家好。大家还好吗？AIE 怎么样？

英文原文

>> Hello. Hi everyone. How's AIE?

太不可思议了。能请到世界上我最喜欢的一些演讲者莅临我们的舞台，在现场与你们——全世界顶尖的一千名 agent 工程师——分享一些内容，一直以来的梦想。我刚刚查看了直播的数据。如果把昨天和今天的全部加在一起，我们在直播上基本上接近 10 万人。所以你们在现场亲眼见证了全球许多人在舞台上收看的内容。Lydia 昨天的演讲，由于技术原因，你们很多人没能看到。我们昨晚发布了，大家可以补看。已经有 30 万人看过了。太疯狂了。绝对太疯狂了。有人想大声说出在整整两天以及两个工作坊中最喜欢的演讲吗？大声说出你们最喜欢的工作坊。有人吗？

英文原文

>> Um, that was incredible. Uh always a dream to have some of my favorite speakers in the world grace our stage share a little bit with you in person uh the thousand top aentic engineers in the world. I just checked the numbers on the live stream. Uh if we add everything together yesterday and today we're basically close to 100,000 people on the live stream. Uh so you have uh you sort of seen live uh what a lot of people are tuning in around the world on stage. Lydia's talk yesterday. Uh a lot of you did not get to see for technical reasons. We launched last night and you can catch up on it. 300,000 people have. Crazy. Absolutely crazy. Uh uh does anyone want to shout out like favorite talks of the the whole two-day and two two and workshops too? Shout out your favorite workshops. Anyone?

Jev。

英文原文

>> Jev.

Jev。是的，非常鼓舞人心。

英文原文

>> Jev. Yeah. Very inspiring.

Potato。

英文原文

>> Potato.

对，Lauren。她会很喜欢的。那个是什么？

英文原文

>> Yes. Lauren. Uh she she'll love that. What's that?

Artcraft。

英文原文

>> Artcraft.

Artcraft。谁打算去重写软件？谁要举手？谁打算去重写一切并开源一切？是的，人群中有很多人举手。多得数不过来。我也想向 Jared Sumner 致敬。Bun AOT 以及许多令人鼓舞的开发工具成果。还有如此庞大的

英文原文

>> Artcraft. Uh you gonna who's going to rewrite software? Like who raise your hands. Who's going to rewrite and open source everything? Yeah, tons tons of you in the crowd. Um I I mean there's too many to name. I want to shout out Jared Sner as well. Uh Bun AOT and and a lot of inspiring uh DevTools work. Uh and just such a huge

感谢赞助商与特别里程碑

社区给予了极大的支持，来自我们的主赞助商 DeepMind。他们今天发布了 Hydrogen。大家今天也觉得很棒吧？是的。还有 OpenAI、Anthropic、Firecrawl，以及在博览区展厅和 Token Billionaire 酒廊的所有其他赞助商。有太多的事情值得庆祝，但有一个特别的里程碑，我非常想在这里庆祝，这是我们只会跨越一次的里程碑。我们只会在这个房间里庆祝。让我们播放视频。

英文原文

u

outpouring of support from the community

from DeepMind our presenting sponsor. Um they launched hydrogen today. Did you guys enjoy that as well? Yeah. um as well as um yeah also open and thropic firecrawl um and all the other sponsors uh down um in the expo floor as well as the token billionaire lounge. Um so there's lots to celebrate um but there's one really particular milestone that I wanted to um uh to to celebrate here that we only ever pass once. We will only celebrate in this room. Let's play the video.

[音乐]

英文原文

>> [music]

[music]

简短而有力。欢迎 AI Engineer 2026 团队，庆祝订阅量突破 100 万！

[掌声]

这就是幕后团队，一直负责制作。我们的总经理 Leah、Natalie、Ilana、Matt、Kyle、Vincent、Sydney、Flo、Jesse，还有后台的几位成员，我们现在不能请他们上台，因为他们必须为活动做收尾。Leah，不知道你是否想说几句，我们想向团队成员颁发一些东西。

英文原文

>> short and sweet. Welcome the AI engineer 2026 team celebrating 1 million subscribers.

[applause]

Um, so this is the team behind the scenes that's been producing uh our GM Leah, uh, Natalie and Ilana, Matt, Kyle, uh, Vincent, Sydney, Flo, Jesse, and a couple more be uh, behind the stage that, uh, we can't bring on stage right now because they have to wrap up the show. Um, Leah, I don't know if you have a few words and we want to pres present some stuff to the team members.

表彰核心成员

当然。就像 Swix 刚才说的，我们在过去一年里实现了巨大的增长，除了我们的系列活动，显然还有我们的 YouTube。这里有两个人，在过去三年里，真的从第一天起就对此起到了绝对关键的作用。我们今天非常想在台上表彰他们两位，所以如果 Flo 和 Vincent 可以走上前来的话，请。

英文原文

>> Absolutely. Um, so as uh, Swix is saying, we've grown dramatically in the last year. um along with our event series obviously our YouTube there's a couple of people here who have been absolutely pivotal to that over the last three years really since day one and we really want to recognize both of them today up on stage so if Flo and Vincent can step forward please

你们将代表 AIE 接受奖牌。好的。

英文原文

>> and you're going to receive the plaques on behalf of AIE all right

Vincent，你还会得到这个小奖励，你可以稍后再打开。巧合的是，除了我之外，他们两位是仅有的在 AIE 发表演讲的团队成员。你们可以在线查看他们的演讲。大家知道是哪个频道。Vincent 从第一天起就和我们在一起。他负责设计、整体视觉风格以及品牌。关于 Vincent 的所有亮点，比如缩略图。如果你曾经点击过 AIE 的视频，那都是因为他。视频上的故障特效，大家很喜欢的所有那些视频，都是他在凌晨两点左右跑来跑去，在图腾柱上更换视频。而且你也有自己的演讲，对吧，在 Worlds 上做的。太棒了，非常感谢你所有的帮助和支持。Flo 是我在发布 YouTube 频道、市场推广方面的搭档。如果你看过那些推文、邮件，以及幕后所有的分析数据，那也全部都是 Flo 做的。同时他还负责维护我们的 NAS，也就是确保我们绝不会丢失任何东西。我们还提供了很多针对演讲者的服务。所以拍摄过的每一张照片都有托管，因为他是个囤积狂。我也是个囤积狂。

英文原文

>> and Vincent you get you get this little bonus as well you can open it later uh so coincident Inidentally, both of them are the uh the only two team members other than me that have spoken at AIE. You can check out their talks online. You know what channel. Um Vincent has been with us since day one. He's responsible for the design, the the look and feel, the brand. Uh anything you enjoyed about Vincent, the thumbnails. Um if you've ever clicked on an AIE video, it's it's because of him. Uh the the glitch effects on the you enjoy that all those those videos, that's him uh running around at like 2 am in the morning changing the videos uh on the on the totems. Um yeah any any and you have your talk right that you did at Worlds. Um cool. Thank you so much for for all the the help and support. Um and Flo uh my partner in crime on publishing the YouTube channel marketing uh if you've seen the tweets and the emails um and all the analytics behind the scenes that that's all flow as well. Uh as well as steady up our NAS which uh means make sure we never lose anything. Um so we're providing a lot of speaker services as well. So every photo ever taken is is hosted because he's a hoarder. I'm a hoarder.

[笑声]

英文原文

>> [laughter]

未来展望与闭幕

这令我的团队感到苦恼。是的，这就是我们的团队。我们一直在积极扩张规模，努力打造一款始终如一的产品。我们最开始是每年举办一场活动，去年是每年三场活动，今年是全年的第四场旗舰活动。到了明年，我们可能大概会保持大体稳定，或许会增加一个较小的第五场。但我们也在拓展大量的合作关系，对吧？我们上周刚举办了 AI 上海，这是在中国的首场 AI 活动。而且我们刚才也宣布了明年举办 AIE 印度。因此将会有更多的合作关系不断发展。很多人问我们如何在自己的城市举办 AIE，直接在网上申请即可，上面有一个合作表格。我想大概就是这些了。非常感谢大家，我们会在周围一起庆祝。我知道你们大家得去参加庆功派对，你们知道该去哪里，去查一下就好。除此之外，祝大家玩得开心，非常感谢大家带来的一届精彩的 AIE 和一个令人难忘的 2026，我们下——

英文原文

>> to the chagrin of my my team. Um yeah, so this is the team. Uh we've brought uh we've really scaled aggressively. We're trying to build a consistent product. Um we started with one event a year last year with three events a year. This year is the fourth uh flagship uh event of the year to next year. We're probably going to stay roughly consistent, maybe add a small uh fifth one. Um but we're also extending a lot of partnerships, right? So uh we just had AI Shanghai last week, the first AI in China. uh as well as uh we're just announced AIE India as well next next year um so there will be a lot more partnerships to grow um and a lot of you have asked us how to run an AIE in your city and uh well just apply online there's there's a partnerships form I think that's about it um thank you so much uh we will be around and uh celebrating you I know you guys got to go to the afterparty you know where to go look it up but otherwise have a great time thank you so much for a wonderful AIE and a fantastic 2026 see you next

[掌声]

英文原文

>> [applause]

我们要拍张照片吗？

Heat。Heat。

That's hey。

英文原文

>> Should we take a photo?

Heat. Heat.

That's hey.