Lenny Rachitsky: Ben, thank you so much for being here. Welcome to the podcast.

列尼·拉奇茨基: Ben，非常感谢你来到这里。欢迎来到我们的播客。

Ben Mann: Thanks for having me. Great to be here, Lenny.

本·曼: 谢谢你的邀请。很高兴能来到这里，Lenny。

AI人才争夺战与指数级进展

Lenny Rachitsky: I want to start with something that's very timely, something that's happening this week. Something that's in the news right now is this whole Zuck coming after all the top AI researchers offering them $100 million signing bonuses, $100 million comp. He's poaching from all the top AI labs. I imagine this something you're dealing with. I'm just curious, what are you seeing inside Anthropic and just what's your take on the strategy? Where do you think things go from here?

列尼·拉奇茨基: 我想从一个非常及时的话题开始，这周正在发生的事情。现在新闻里到处都是扎克伯格追逐顶尖AI研究人员的消息，为他们提供上亿美元的签约奖金和薪酬。他正在从所有顶尖的AI实验室挖人。我想这是你正在处理的事情。我很好奇，你在Anthropic内部看到了什么？你对这个策略有什么看法？你认为接下来会发生什么？

Ben Mann: Yeah, I mean I think this is a sign of the times. The technology that we're developing is extremely valuable. Our company is growing super, super fast. Many of the other companies in the space are growing really fast. And at Anthropic, I think we've been maybe much less affected than many of the other companies in the space because people here are so mission oriented and they stay because... They get these offers and then they say, "Well, of course I'm not going to leave because my best case scenario at Meta is that we make money and my best case at Anthropic is we affect the future of humanity and try to make AI flourish and human flourishing go well." To me, it's not a hard choice. Other people have different life circumstances and it makes it a much harder decision for them. For anybody who does get those mega offers and accepts them, I can't say I hold it against them when they accept it, but it's definitely not something that I would want to take myself if it came to me.

本·曼: 是的，我认为这是时代的一个标志。我们正在开发的技术非常有价值。我们的公司正在超高速发展。这个领域的许多其他公司也在快速增长。而在Anthropic，我认为我们受到的影响可能比这个领域的许多其他公司小得多，因为这里的人都非常以使命为导向，他们留下来是因为……他们收到了这些报价，然后他们说：“嗯，我当然不会离开，因为我在Meta最好的情况是赚钱，而我在Anthropic最好的情况是影响人类的未来，努力让AI蓬勃发展，促进人类的繁荣。”对我来说，这不是一个艰难的选择。其他人有不同的生活环境，这让他们做出决定要困难得多。对于任何确实收到那些巨额报价并接受的人，当他们接受时，我不能说我对此有任何怨言，但如果这个机会给我，我肯定不会接受。

Lenny Rachitsky: In terms of the offers do you think, is this a real number that you're seeing this $100 million signing bonus, is that a real thing? I don't know if you've actually seen that.

列尼·拉奇茨基: 关于这些报价，你认为你看到的这个1亿美元的签约奖金是真的吗？这个数字真实存在吗？我不知道你是否真的见过。

Ben Mann: I'm pretty sure it's real. If you just think about the amount of impact that individuals can have on a company's trajectory, in our case, we are selling hotcakes and if we get a 1 or 10 or 5% efficiency bonus on our inference stack, that is worth an incredible amount of money. And so to pay individuals like $100 million over four year package, that's actually pretty cheap compared to the value created for the business. I think we're just in an unprecedented era of scale and it's only going to get crazier actually. If you extrapolate the exponential on how much companies are spending, it's like 2X a year roughly in terms of CapEx, and today we're maybe in the globally $300 billion range, the entire industry spending on this, and so numbers like 100 million are a drop in the bucket. But if you go a few years out, a couple more doublings, we're talking about trillions of dollars and at that point it's just really hard to think about these numbers.

本·曼: 我很确定是真的。如果你想一下个人能对公司发展轨迹产生多大的影响，就我们而言，我们的产品非常畅销，如果我们的推理堆栈效率提升1%、10%或5%，那将价值连城。所以，给个人支付一个四年期1亿美元的薪酬包，与为业务创造的价值相比，这实际上是相当便宜的。我认为我们正处在一个前所未有的规模时代，而且实际上只会变得更加疯狂。如果你推断公司支出的指数级增长，资本支出大约是每年2倍，今天我们全球整个行业在这上面的支出可能在3000亿美元的范围内，所以像1亿这样的数字只是九牛一毛。但如果你再过几年，再翻几番，我们谈论的将是数万亿美元，到那时，这些数字真的很难想象了。

Lenny Rachitsky: Along these lines, something that a lot of people feel with AI progress is that we're hitting plateaus in many ways that it feels like newer models are just not as smart as previous leaps. But I know you don't believe this. I know you don't believe that we've hit plateaus on scaling laws. Talk about just what you're seeing there and what you think people are missing.

列尼·拉奇茨基: 与此相关的是，很多人对AI的进展感觉是，我们在很多方面都遇到了瓶颈，感觉新模型并不像之前的飞跃那样智能。但我知道你不相信这一点。我知道你不认为我们在缩放法则（Scaling Laws：指模型性能随计算、数据和参数规模增加而可预测地提升的经验性规律）上遇到了瓶颈。谈谈你在这方面看到了什么，以及你认为人们忽略了什么。

Ben Mann: It's kind of funny because this narrative comes out every six months or so and it's never been true, and so I kind of wish people would have a little bit of a bullshit detector in their heads when they see this. I think progress has actually been accelerating where if you look at the cadence of model releases, it used to be once a year and now with the improvements in our post-training techniques, we're seeing releases every month or three months, and so I would say progress is actually accelerating in many ways, but there's this weird time compression effect. Dario compared it to being in a near light speed journey where a day that passes for you is like five days back on earth and we're accelerating. The time dilation is increasing. And I think that's part of what's causing people to say that progress is slowing down, but if you look at the scaling laws, they're continuing to hold true. We did kind of need this transition from normal pre-training to reinforcement learning scaling up to continue the scaling laws, but I think it's kind of like for semiconductors where it's less about the density of transistors that you can fit on a chip and more about how many flops can you fit in a data center or something. You have to change the definition around a little bit to keep your eye on the prize. But yeah, this is one of the few phenomena in the world that has held across so many orders of magnitude. It's actually pretty surprising that it is continuing to hold. To me, if you look at fundamental laws of physics, many of them don't hold across 15 orders of magnitude, so it's pretty surprising.

本·曼: 这有点好笑，因为这种说法每六个月左右就会出现一次，但它从来都不是真的，所以我有点希望人们在看到这种说法时，脑子里能有个“扯淡探测器”。我认为进展实际上一直在加速，如果你看看模型发布的节奏，以前是一年一次，现在随着我们后期训练技术的改进，我们看到每个月或每三个月就有新版本发布，所以我想说，进展在很多方面实际上是在加速，但有一种奇怪的时间压缩效应。达里奥曾把它比作一次近光速旅行，你度过的一天就像地球上的五天，而且我们还在加速。时间膨胀效应正在增加。我认为这是导致人们说进展正在放缓的部分原因，但如果你看看缩放法则，它们仍然成立。我们确实需要从普通的预训练过渡到强化学习的规模化，以延续缩放法则，但我认为这有点像半导体，关键不在于你可以在一个芯片上集成多少晶体管，而在于你可以在一个数据中心里容纳多少浮点运算能力。你必须稍微改变一下定义，才能抓住重点。但是，是的，这是世界上为数不多的在如此多数量级上都成立的现象之一。它能持续成立，其实相当令人惊讶。对我来说，如果你看看物理学的基本定律，很多定律都无法在15个数量级上成立，所以这相当惊人。

Lenny Rachitsky: What you're saying essentially is we're seeing newer models being released more often, and so we're comparing it to the last version and we're just not seeing as much advance. But if you go back and it was like a model released once a year, it was a huge leap, and so people are missing that. We're just seeing many more iterations.

列尼·拉奇茨基: 你说的基本上是，我们看到新模型发布得更频繁了，所以我们把它和上一个版本比较，感觉进步没那么大。但如果你回想一下，以前模型一年发布一次，那是一个巨大的飞跃，所以人们忽略了这一点。我们只是看到了更多的迭代。

Ben Mann: I guess, to be a little bit more generous to the people saying things are slowing down. I think that for some tasks we are saturating the amount of intelligence needed for that task, maybe to extract information from a simple document that already has form fields on it or something like it's just so easy that okay, yeah, we're already at 100% and there's this great chart on Our World in Data that shows that when you release a new benchmark within six to 12 months, it immediately gets saturated. And so maybe the real constraint is how can we come up with better benchmarks and better ambition of using the tools that then reveals the bumps in intelligence that we're seeing now.

本·曼: 我想，对那些说事情正在放缓的人更宽容一点来看。我认为对于某些任务，我们正在饱和完成该任务所需的智能量，也许是从一个已经有表单字段的简单文档中提取信息，或者类似的事情，这太容易了，所以，是的，我们已经达到100%了。在Our World in Data上有一张很棒的图表显示，当你发布一个新的基准测试时，在六到十二个月内，它会立刻饱和。所以也许真正的限制是我们如何能提出更好的基准和更有雄心地使用这些工具，从而揭示我们现在看到的智能上的飞跃。

定义AGI及其经济影响

Lenny Rachitsky: That's a good segue to you have a very specific way of thinking about AGI and defining what AGI means.

列尼·拉奇茨基: 这是一个很好的引子，你对AGI（通用人工智能：Artificial General Intelligence，指具备与人类同等智慧，或超越人类智慧，可以执行人类任何智力任务的AI）有一个非常具体的思考方式和定义。

Ben Mann: I think AGI is kind of a loaded term, and so I tend not to use it very much anymore internally. Instead, I like the term transformative AI because it's less about can it do as much as people do? Can it do literally everything and more about objectively is it causing transformation in society and the economy? A very concrete way of measuring that is the Economic Turing Test. I didn't come up with this, but I really like it. It's this idea that if you contract an agent for a month or three months on a particular job, if you decide to hire that agent and it turns out to be a machine rather than a person, then it's passed the Economic Turing Test for that role. And then you can sort of expand that out in the same way that for measuring purchasing power parity or inflation, there's a basket of goods. You can have a market basket of jobs, and if the agent can pass the Economic Turing Test for 50% of money-weighted jobs, then we have transformative AI and the exact thresholds don't really matter that much, but it's kind of illustrative to say if we pass that threshold, then we would expect massive effects on world GDP increases and societal change and how many people are employed and things like that because societal institutions and organizations are sticky, it's slow to have change, but once these things are possible you know that it's the start of a new era.

本·曼: 我认为AGI这个词有点含混不清，所以我在内部已经不怎么用了。我更喜欢“变革性AI”这个词，因为它更少关注它是否能做和人一样多的事，是否能做所有事，而更多地关注它是否客观上在社会和经济中引起了变革。一个非常具体的衡量方法是经济图灵测试（Economic Turing Test：一种衡量AI能力的标准，如果一个AI在一个特定工作岗位上表现得与人类无法区分，以至于雇主愿意雇佣它，那么它就通过了该岗位的经济图灵测试）。这不是我提出的，但我非常喜欢它。这个想法是，如果你为一个特定的工作雇佣一个代理一个月或三个月，如果你决定雇佣那个代理，结果发现它是一台机器而不是一个人，那么它就通过了那个角色的经济图灵测试。然后你可以把它扩展开来，就像衡量购买力平价或通货膨胀时有一篮子商品一样。你可以有一个工作市场篮子，如果那个代理能通过50%按金钱加权的工作的经济图灵测试，那么我们就拥有了变革性AI，确切的阈值其实不那么重要，但它很有说明性，表明如果我们越过那个阈值，我们就会预期到对世界GDP增长、社会变革以及就业人数等方面产生巨大影响，因为社会制度和组织是粘性的，变革很慢，但一旦这些事情成为可能，你就知道这是一个新时代的开始。

Lenny Rachitsky: Along these lines, Dario, your CO recently talked about how AI is going to take a huge part of, I don't know, half of white-collar jobs, that unemployment might go up to something like 20%. I know you're even more vocal and opinionated about just how much impact AI is already having in the workplace that people may not even be realizing. Talk about just what you think people are missing about the impact AI is going to have on jobs and is already having.

列尼·拉奇茨基: 与此相关，你的首席执行官达里奥最近谈到，人工智能将占据很大一部分，我不知道，也许是一半的白领工作，失业率可能会上升到20%左右。我知道你对人工智能已经对工作场所产生的影响，以及人们可能甚至没有意识到的影响，更加直言不讳和有主见。谈谈你认为人们对于人工智能将对工作产生的影响以及已经产生的影响，有哪些误解。

Ben Mann: Yeah, so from an economic standpoint, there's a couple different kinds of unemployment, and one is because the workers just don't have the skills to do the kinds of jobs that the economy needs. And another kind is where those jobs are just completely eliminated, and I think it's going to be actually a combination of these things, but if you just think about 20 years in the future where we're way past the singularity, it's hard for me to imagine that even capitalism will look at all it looks today. If we do our jobs, we will have safe aligned superintelligence, we'll have, as Dario says, in Machines of Love and Grace, a country of geniuses in a data center, and the ability to accelerate positive change in science, technology, education, mathematics, it's going to be amazing. But that also means in a world of abundance where labor is almost free and anything you want to do, you can just ask an expert to do for you, then what do jobs even look like? And so I guess there's this scary transition period from where we are today where people have jobs and capitalism works and the world of 20 years from now where everything is completely different, but part of the reason they call it the singularity is that it's a point beyond which you can't easily forecast what's going to happen. It's just such a fast rate of change and so different that it's hard to even imagine. I guess taking the view from the limit, it's pretty easy to say hopefully we'll have figured it out. And in a world of abundance, maybe the jobs themselves, it's not that scary, and I think making sure that that transition time goes well is pretty important.

本·曼: 是的，从经济角度来看，有几种不同类型的失业，一种是因为工人根本不具备经济所需要的工作技能。另一种是那些工作被完全淘汰了，我认为这将是这些因素的结合。但如果你只思考20年后的未来，那时我们已经远远超过了奇点（Singularity：一个假设的时间点，技术增长变得不可控和不可逆，导致人类文明发生不可预见的剧变），我很难想象资本主义还会像今天这样。如果我们做好我们的工作，我们将拥有安全对齐的超级智能，我们将拥有，正如达里奥在《爱与恩典的机器》中所说，一个数据中心里的天才国度，以及加速科学、技术、教育、数学等领域积极变革的能力，那将是惊人的。但这也意味着，在一个富足的世界里，劳动力几乎是免费的，你想做的任何事，你都可以请一个专家为你做，那么工作会是什么样子呢？所以我想，从我们今天人们有工作、资本主义运转的世界，到20年后一切都完全不同的世界，有一个可怕的过渡期。但人们称之为奇点，部分原因在于它是一个你无法轻易预测之后会发生什么的临界点。变化的速度如此之快，如此之不同，以至于很难想象。我想从极限的角度看，很容易说希望我们能解决好。在一个富足的世界里，也许工作本身并不那么可怕，我认为确保过渡期顺利进行非常重要。

Lenny Rachitsky: There's a couple of threads I want to follow there. One is people hear this, there's a lot of headlines around this. Most people probably don't actually feel this yet or see this happening and so there's always this, I guess, I don't know, maybe, but I don't know it's hard to believe, my job seems fine. Nothing's changed. What are you seeing just happening today already that you think people don't see or misunderstand in terms of the impact AI is having on jobs?

列尼·拉奇茨基: 我想跟进几个线索。一个是人们听到这个，有很多关于这个的头条新闻。大多数人可能还没有真正感觉到或者看到这种情况发生，所以总是有这种，我猜，我不知道，也许吧，但我不知道，这很难相信，我的工作看起来还不错。什么都没变。就今天已经发生的事情而言，你认为人们没有看到或者误解了AI对工作的影响是什么？

Ben Mann: I think part of this is that people are really bad at modeling exponential progress. And if you look at an exponential on a graph, it looks flat and almost zero at the beginning of it, and then suddenly you hit the knee of the curve and things are changing real fast and then it goes vertical. That's the plot that we've been on for a long time. I guess I started feeling it in 2019 maybe when GPT-2 came out and I was like, "Oh, this is how we're going to get to AGI." But I think that was pretty early compared to a lot of people where when they saw ChatGPT, they were like, "Wow, something is different and changing." And so I guess I wouldn't expect widespread transformation in a lot of parts of society, and I would expect this skepticism reaction. I think it's very reasonable and it's exactly what is the standard linear view of progress. But I guess to cite a couple of areas where I think things are changing quite quickly. In customer service we're seeing with things like Fin and Intercom, they're a great partner of ours, 82% customer service resolution rates automatically without a human involved. And in terms of software engineering, our Claude Code team, like 95% of the code is written by Claude. But I think a different way to phrase that is that we write 10X more code or 20X more code, and so a much, much smaller team can just be much, much more impactful. And similarly for the customer service, yes, you can phrase it as 82% customer service resolution rates, but that nets out in the humans doing those tasks, able to focus on the harder parts of those tasks. And for the more tricky situations that in a normal world like five years ago, they would've had to just drop those tickets because it was too much effort for them to actually go do the investigation. There were too many other tickets for them to worry about. I think in the immediate term, there will be a massive expansion of the pie and the amount of labor that people can do. I've never met a hiring manager at a growth company and heard them say, "I don't want to hire more people." That's the hopeful version of it. But with things that are lower skill jobs or less headroom on how good they can be, I think there will be a lot of displacement. It is just something we as a society need to get ahead of and work on.

本·曼: 我认为部分原因是人们非常不擅长模拟指数级增长。如果你看图表上的指数曲线，它在开始时看起来是平的，几乎为零，然后突然你到达了曲线的拐点，事情变化得非常快，然后它就垂直上升了。这就是我们长期以来的轨迹。我想我可能是在2019年GPT-2问世时开始感觉到这一点的，当时我就想，“哦，这就是我们通往AGI的方式。”但我认为这比很多人要早，当他们看到ChatGPT时，他们才觉得，“哇，有些东西不同了，正在改变。”所以我猜我不会期望社会很多部分会发生广泛的变革，而且我会预料到这种怀疑的反应。我认为这非常合理，也正是标准的线性进步观。但我想举几个我认为变化非常快的领域。在客户服务领域，我们看到像Fin和Intercom这样的产品，它们是我们的重要合作伙伴，实现了82%的客户服务问题自动解决率，无需人工介入。在软件工程方面，我们的Claude Code团队，大约95%的代码是由Claude编写的。但我认为换一种说法是，我们编写了10倍或20倍的代码，所以一个规模小得多的团队可以产生大得多的影响。同样对于客户服务，是的，你可以说82%的客户服务问题解决率，但这最终意味着从事这些任务的人能够专注于这些任务中更困难的部分。对于那些在五年前的正常世界里，他们可能不得不放弃的更棘手的工单，因为投入调查的精力太大了。有太多其他的工单需要他们担心。我认为在短期内，劳动力市场和人们可以做的工作量将会有一个巨大的扩展。我从未见过一个成长型公司的招聘经理说，“我不想再招更多的人了。”这是充满希望的版本。但对于那些技能要求较低或提升空间不大的工作，我认为会有大量的岗位被取代。这正是我们作为一个社会需要提前应对和努力解决的问题。

为AI驱动的未来做准备

Lenny Rachitsky: I want to talk more about that, but something that I also want to help people with is how do they get a leg up in this future world? They listen to this, they're like, "Oh, this doesn't sound great. I need to think ahead." I know you won't have all the answers, but just do you have any advice for folks that want to try to get ahead of this and kind of future-proof their career and their life to not be replaced by AI? Anything you've seen people do, anything you recommend they start trying to do more of?

列尼·拉奇茨基: 我想更多地讨论这一点，但我也想帮助人们，他们如何在这个未来的世界中占得先机？他们听到这个，会觉得，“哦，这听起来不太好。我需要提前考虑。”我知道你不会有所有的答案，但你有什么建议给那些想提前应对，为自己的职业和生活做好未来规划，以避免被AI取代的人吗？你见过人们做过什么，或者你建议他们开始多做些什么？

Ben Mann: Even for me and being in the center of a lot of this transformation, I'm not immune to job replacement either. Just some vulnerability there of at some point it's coming for all of us.

本·曼: 即使对我来说，身处这场变革的中心，我也不能幸免于工作被取代。这里透露一点脆弱，那就是总有一天，我们所有人都将面临这个问题。

Lenny Rachitsky: Even you, Ben, now.

列尼·拉奇茨基: 即使是你，Ben，现在也是。

Ben Mann: And you, Lenny.

本·曼: 还有你，Lenny。

Lenny Rachitsky: And me.

列尼·拉奇茨基: 还有我。

Ben Mann: Oh, wait, we've gone too far now. Okay. But in terms of the transition period, yeah, I think there are things that we can do, and I think a big part of it is just being ambitious and how you use the tools and being willing to learn new tools. People who use the new tools as if they were old tools tend to not succeed. As an example of that, when you're coding, people are very familiar with autocomplete, people are familiar with SimpleChat where they can ask questions about the code base, but the difference between people who use Claude Code very effectively and people who use it not so effectively is like are they asking for the ambitious change? And if it doesn't work the first time, asking three more times because our success rate when you just completely start over and try again is much, much higher than if you just try once and then just keep banging on the same thing that didn't work. And even though that's a coding example and coding is one of the areas that's taking off most dramatically, we have seen internally that our legal team and our finance team are getting a ton of value out of using Claude Code itself. We're going to be making better interfaces so that they will have an easier time and require a little bit less jumping in the deep end of using Claude Code in the terminal. But yeah, we're seeing them use it to redline documents and use it to run BigQuery analyses of our customers and our revenue metrics. I guess it's about taking that risk and even if it feels like a scary thing, trying it out.

本·曼: 哦，等等，我们现在说得太远了。好吧。但在过渡期方面，是的，我认为有些事情我们可以做，很大一部分就是在使用这些工具时要有雄心，并且愿意学习新工具。那些把新工具当作旧工具来用的人往往不会成功。举个例子，在编程时，人们非常熟悉自动完成，也熟悉可以询问代码库问题的简单聊天功能，但高效使用Claude Code的人和不那么高效的人之间的区别在于，他们是否在要求进行雄心勃勃的改变？如果第一次不成功，他们会再尝试三次，因为当你完全重新开始再试一次时，我们的成功率要比你只尝试一次然后一直纠结于那个行不通的方法高得多。尽管这是一个编程的例子，编程是发展最快的领域之一，但我们在内部看到，我们的法务团队和财务团队在使用Claude Code本身时也获得了巨大的价值。我们将制作更好的界面，让他们用起来更容易，不需要那么费劲地跳入在终端中使用Claude Code的深渊。但是，是的，我们看到他们用它来批注文档，用它来运行BigQuery分析我们的客户和收入指标。我想这关乎于承担风险，即使感觉很可怕，也要尝试一下。

Lenny Rachitsky: Okay, so the advice here is use the tools. That's something everyone's always saying, just actually use these tools. It's like sit in Claude Code. And your point about being more ambitious than you naturally feel like being because maybe it'll actually accomplish the thing. This tip of trying it three times so the idea there is it may not get it right the first time. Is the tip there ask it in different ways or is it just try harder, try again?

列尼·拉奇茨基: 好的，所以这里的建议是使用工具。这是每个人一直在说的事情，就是真正地去使用这些工具。就像坐下来用Claude Code。你说的要比你自然感觉的更有雄心，因为也许它真的能完成那件事。这个“试三次”的技巧，意思是它第一次可能做不对。这个技巧是说用不同的方式问它，还是说更努力地再试一次？

Ben Mann: Yeah, I mean you can just literally ask the exact same question. These things are stochastic and sometimes they'll figure it out and sometimes they won't. In every one of these model cards, it always shows pass it one versus pass it in. And that's exactly the thing where they try the exact same prompt, sometimes it gets it, sometimes it doesn't. That's the dumbest advice. But yeah, I think if you want to be a little bit smarter about it, there can be gains there of saying, "Here's what you already tried and it didn't work, so don't try that. Try something different." That can also help.

本·曼: 是的，我的意思是你可以直接问完全相同的问题。这些东西是随机的，有时候它们能解决，有时候不能。在每一个模型卡片里，总会显示一次通过率和多次通过率。这正是他们尝试完全相同的提示，有时成功，有时不成功的情况。这是最笨的建议。但是，是的，我想如果你想更聪明一点，可能会有收获，比如告诉它，“这是你已经试过的，但没用，所以别试那个了。试试别的。”这也能有所帮助。

Lenny Rachitsky: The advice is comes back to something that a lot of people talk about these days is you won't be replaced by AI at least anytime soon you'll be replaced by someone that is very good using AI?

列尼·拉奇茨基: 你的建议回到了很多人最近都在谈论的一个观点，那就是你不会被AI取代，至少短期内不会，你会被那些非常擅长使用AI的人取代？

Ben Mann: I think in that area it's more like your team will just do dramatically more stuff. We're definitely not slowing down on hiring at all, and some people are confused by that. Even in an onboarding class, somebody asked that and they were like, "Why did you hire me if we're all just going to be replaced?" And the answer is the next couple of years are really critical to get right and we're not at the point where we're doing complete replacement. Like I said, we're still at that flat zero looking part of the exponential compared to where we will be. It is super important to have great people and that's why we're hiring super aggressively.

本·曼: 我认为在这个领域，更像是你的团队会做更多的事情。我们绝对没有放慢招聘的步伐，有些人对此感到困惑。甚至在一个入职培训班上，有人问到这个问题，他们说：“既然我们最终都会被取代，你们为什么还要雇佣我？”答案是，接下来的几年对于走上正轨至关重要，我们还没有到完全替代的阶段。就像我说的，与未来相比，我们现在还处于指数曲线那看似平坦为零的部分。拥有优秀的人才非常重要，这就是我们积极招聘的原因。

Lenny Rachitsky: Let me take another approach to asking this question something ask everyone that's at the very cutting edge of where AI is going. You have kids, knowing what you know about where AI is heading and all these things you've been talking about, what are you focusing on teaching your kids to help them thrive in this AI future?

列尼·拉奇茨基: 我换个方式问这个问题，这个问题我问过每个处在AI最前沿的人。你有孩子，知道你所知道的关于AI未来的发展方向以及你一直在谈论的这些事情，你注重教给你的孩子什么，来帮助他们在这个AI未来中茁壮成长？

Ben Mann: Yeah, I have two daughters, a one-year-old and a three-year-old, so it's pretty in the basics still. And our three-year-old is now capable of just conversing with Alexa Plus and asking her to explain stuff and play music for her and all that stuff. She's been loving that. But I guess more broadly, she goes to a Montessori school and I just love the focus on curiosity and creativity and self-led learning that Montessori has. I guess if I were in a normal era like 10, 20 years ago and I had a kid, maybe I would be trying to line her up for going to a top tier school and doing all the extracurriculars and all that stuff. But at this point, I don't think any of it's going to matter. I just want her to be happy and thoughtful and curious and kind. And the Montessori school is definitely doing great at that. They text us throughout the day. Sometimes they're like, "Oh, your kid got in an argument with this other kid and she has really big emotions and she tried to use her words." I love that. I think that's exactly the kind of education that I think is most important, that the facts are going to fade into the background.

本·曼: 是的，我有两个女儿，一个一岁，一个三岁，所以现在还处于非常基础的阶段。我们三岁的女儿现在已经能够和Alexa Plus对话，让她解释东西、放音乐给她听等等。她非常喜欢这个。但更广泛地说，她去的是一所蒙特梭利学校，我非常喜欢蒙特梭利对好奇心、创造力和自主学习的关注。我想如果我在一个正常的时代，比如10年、20年前，我有一个孩子，我可能会努力让她进入顶尖学校，参加各种课外活动等等。但现在，我认为这些都不重要了。我只希望她快乐、有思想、好奇和善良。蒙特梭利学校在这方面做得非常好。他们白天会给我们发信息。有时他们会说，“哦，你的孩子和另一个孩子吵架了，她情绪很激动，但她试着用语言来表达。”我喜欢这个。我认为这正是我认为最重要的教育，事实会淡出背景。

Anthropic的诞生：以安全为核心

Lenny Rachitsky: I want to go back to the beginning of Anthropic. Famously you and eight of you left OpenAI back in the day in 2020, I believe the end of 2020 to start Anthropic. Talk a little bit about why this happened, what you guys saw. I'm curious, just if you're willing to share more, just what is it that you saw at OpenAI, what'd you experience there that made you feel like, okay, we got to go do our own thing?

列尼·拉奇茨基: 我想回到Anthropic的起点。众所周知，你和另外八个人在2020年底离开了OpenAI，创办了Anthropic。谈谈这背后的原因，你们看到了什么。我很好奇，如果你愿意分享更多，你在OpenAI看到了什么，经历了什么，让你们觉得，好吧，我们得自己干了？

Ben Mann: Yeah, so for the listeners, I was part of the GPT-2=3 project at OpenAI, ended up being one of the first authors on the paper, and I also did a bunch of demos for Microsoft to help raise $1 billion from them, did the tech transfer of GPT-3 to their systems so that they could help serve the model in Azure. I did a bunch of different things there on both the more researchy side and the product side. One weird thing about OpenAI is that while I was there, Sam talked about having three tribes that needed to be kept in check with each other, which was the safety tribe, the research tribe, and the startup tribe. And whenever I heard that, it just struck me as the wrong way to approach things because the company's mission apparently is to make the transition to AGI safe and beneficial for humanity. And that's basically the same as Anthropic's mission. But internally, it felt like there was so much tension around these things. And I think when push came to shove, we felt like safety wasn't the top priority there. And there are good reasons that you might think that if you thought safety was going to be easy to solve or if you thought it wasn't going to have a big impact, or if you thought that the chance of big negative outcomes was vanishingly small, then maybe you would just do those kinds of actions. But at Anthropic we felt, I mean we didn't exist then, but it was basically the leads of all the safety teams at OpenAI, we felt that safety is really important, especially on the margin. And so if you look at who in the world is actually working on safety problems, it's pretty small set of people. Even now, I mean the industry is blowing up, as I mentioned, 300 billion a year CapEx today, and I would say maybe less than 1,000 people working on it worldwide, which is just crazy. That was fundamentally why we left. We felt like we wanted an organization where we could be on the frontier, we could be doing the fundamental research, but we could be prioritizing safety ahead of everything else. And I think that's really panned for us in a surprising way. We didn't know even if it would be possible to make progress on the safety research because at the time, we had tried a bunch of safety through debate and the models weren't good enough. And so we basically had no results on all of that work, and now that exact technique is working and many others that we have been thinking about for a long time. Yeah, fundamentally it comes down to is safety the number one priority? And then something that we've sort of tacked on since then is like, can you have safety and be at the front here at the same time? And if you look at something like sycophancy, I think Claude is one of the least sycophantic models because we've put so much effort into actual alignment and not just trying to good heart our metrics of saying user engagement is number one, and if people say yes, then it's good for them.

本·曼: 是的，给听众们介绍一下，我曾是OpenAI的GPT-2和GPT-3项目的一员，最终成为论文的首批作者之一，我还为微软做了一系列演示，帮助他们筹集了10亿美元，并把GPT-3的技术转移到他们的系统中，这样他们就可以在Azure中提供模型服务。我在那里做了很多不同的事情，既有更偏研究的，也有偏产品的。关于OpenAI，有一件奇怪的事是，当我在那里的时候，萨姆（Sam Altman）谈到需要保持三个部落之间的平衡，那就是安全部落、研究部落和创业部落。每当我听到这个，我就觉得这是一种错误的处理方式，因为公司的使命显然是让向AGI的过渡对人类来说是安全和有益的。这和Anthropic的使命基本相同。但在内部，感觉在这些事情上有很多紧张关系。而且我认为，当关键时刻到来时，我们觉得安全在那里并不是最高优先级。如果你认为安全问题很容易解决，或者你认为它不会产生重大影响，或者你认为出现重大负面结果的可能性微乎其微，那么你可能会采取那样的行动，这是有道理的。但在Anthropic，我们当时觉得，虽然我们那时还不存在，但基本上是OpenAI所有安全团队的负责人，我们觉得安全真的非常重要，尤其是在边缘情况下。所以如果你看看世界上谁真正在研究安全问题，那是一小部分人。即使是现在，我的意思是，这个行业正在爆炸式增长，正如我提到的，今天每年的资本支出是3000亿美元，而我会说全球可能只有不到1000人在研究这个问题，这简直是疯了。这基本上就是我们离开的原因。我们觉得我们想要一个组织，在这里我们可以走在前沿，可以做基础研究，但可以把安全放在其他一切之上。我认为这对我们来说，以一种令人惊讶的方式取得了成功。我们甚至不知道在安全研究上是否可能取得进展，因为当时，我们尝试了很多通过辩论来实现安全的方法，但模型还不够好。所以我们基本上在那项工作上没有任何成果，但现在，那种技术正在奏效，还有许多我们思考了很久的其他技术也在奏效。是的，归根结底，问题在于安全是否是第一要务？然后我们后来又加上了一点，那就是，你能在保持安全的同时，处于行业前沿吗？如果你看看像“谄媚”这样的问题，我认为Claude是“谄媚”程度最低的模型之一，因为我们投入了大量的精力在真正的对齐（Alignment：指确保AI系统的目标和行为与人类的价值观和意图相符的过程）上，而不仅仅是试图粉饰我们的指标，说用户参与度是第一位的，如果人们说是，那么对他们来说就是好的。

安全与能力的相互作用

Lenny Rachitsky: Okay. Let's talk about this tension that you mentioned, this tension between safety and progress, being competitive in the marketplace. I know you spent a lot of your time on safety. I know that as you just alluded to, this is a core part of how you think about AI. I want to talk about why that is, but first of all, just how do you think about this tension between focusing on safety while also not falling way behind?

列尼·拉奇茨基: 好的。让我们谈谈你提到的这种紧张关系，即安全与进步、在市场中保持竞争力之间的紧张关系。我知道你花了很多时间在安全上。我知道正如你刚才提到的，这是你思考AI的核心部分。我想谈谈为什么会这样，但首先，你如何看待在专注于安全的同时又不会远远落后的这种紧张关系？

Ben Mann: Yeah, so initially we thought that it would be sort of one or the other, but I think since then we've realized that it's actually kind of convex in the sense that working on one helps us with the other thing. Initially when Opus 3 came out and we were finally at the frontier of model capabilities, one of the things that people really loved about it was the character and the personality. And that was directly a result of our alignment research. Amanda Askell did a ton of work on this and as well as many others who tried to figure out what does it mean for an agent to be helpful, honest, and heartless, and what does it mean to be in difficult conversations and show up effectively? How do you do a refusal that doesn't shut the person down, but makes them feel like they understand why the agent said, "I can't help you with that. Maybe you should talk to a medical professional, or maybe you should consider not trying to build bio-weapons or something like that." Yeah, I guess that's part of it. And then another piece that's come out is constitutional AI, where we have this list of natural language principles that leads the model to learn how we think a model should behave. And they've been taken from things like the UN Declaration of Human Rights and Apple's privacy terms of service and a whole bunch of other places, many of which we've just generated ourselves that allow us to take a more principled stance, not just leaving it to whatever human raiders we happen to find, but we ourselves deciding what should the values of this agent be? And that's been really valuable for our customers because they can just look at that list and say like, "Yep, these seem right. I like this company, I like this model. I trust it."

本·曼: 是的，最初我们认为这可能是非此即彼的选择，但后来我们意识到，这实际上是凸函数的关系，也就是说，致力于一方面有助于我们在另一方面取得进展。最初当Opus 3问世，我们终于站在了模型能力的前沿时，人们非常喜欢它的一点是它的特性和个性。这直接是我们对齐研究的结果。Amanda Askell在这方面做了大量的工作，还有许多其他人也试图弄清楚，一个代理要怎样才算是有帮助、诚实和无害的，以及在困难的对话中如何有效地表现。你如何做出拒绝，既不会让对方感到被排斥，又能让他们明白为什么代理会说，“我不能帮你这个。也许你应该咨询医疗专业人士，或者也许你应该考虑不要试图制造生物武器之类的。”是的，我想这是其中一部分。然后另一个成果是宪法AI（Constitutional AI：一种训练AI模型的方法，通过一系列明确的原则（宪法）来指导模型的行为，使其更安全、更符合人类价值观，而无需持续的人工监督），我们有一系列自然语言原则，引导模型学习我们认为模型应该如何行为。这些原则取自《联合国人权宣言》、苹果的隐私服务条款以及其他很多地方，其中很多是我们自己生成的，这让我们能够采取更有原则的立场，而不仅仅是依赖于我们碰巧找到的人类评分员，而是我们自己决定这个代理应该具备什么样的价值观。这对我们的客户来说非常有价值，因为他们可以看看那个列表，然后说，“是的，这些看起来是对的。我喜欢这家公司，我喜欢这个模型。我信任它。”

Lenny Rachitsky: Okay, this is awesome. One nugget there is your point that the personality of Claude, its personality is directly aligned with safety. I don't think a lot of people think about that. And this is because of the values that you imbue, is that the word, with constitutional AI and things like that. Like the actual personality of the AIs directly connected to your focus on safety.

列尼·拉奇茨基: 好的，这太棒了。其中一个亮点是，你指出Claude的个性，它的个性与安全直接相关。我不认为很多人会这么想。这是因为你们通过宪法AI等方式赋予了它价值观，是这个词吗？就像AI的实际个性与你们对安全的关注直接相关。

Ben Mann: That's right. That's right. And from a distance, it might seem quite disconnected, like how is this going to prevent X risk? But ultimately it's about the AI understanding what people want and not what they say. We don't want the Monkey Paw Scenario of the genie gives these three wishes and then you end up having everything you touch turns of gold. We want the AI to be like, oh, obviously what you really meant was this, and that's what I'm going to help you with. I think it is really quite connected.

本·曼: 是的，没错。从远处看，这可能显得很脱节，比如这怎么能预防X风险（X-risk：Existential Risk的缩写，指生存风险，即可能导致人类灭绝或永久性地、严重地削弱人类发展潜力的事件）？但归根结底，这是关于AI理解人们想要什么，而不仅仅是他们说了什么。我们不希望出现“猴爪”情景，就是那个精灵给了三个愿望，结果你碰到的所有东西都变成了金子。我们希望AI能像这样，哦，显然你真正的意思是这个，这才是我要帮你的。我认为这确实很有关联。

Lenny Rachitsky: Talk a bit more about this constitutionally AI. This is essentially you bake in, here's the rules that we want you to abide by and it's values, you said it's the Geneva Human Rights Code, things like that. How does that actually work? I think the core here is just this is baked into the model. It's not something you add on top later.

列尼·拉奇茨基: 再多谈谈这个宪法AI。这基本上是你们内置的，这是我们希望你遵守的规则，而且是价值观，你说过是日内瓦人权法典之类的东西。这实际上是怎么运作的？我认为这里的核心是，这是内置在模型里的。不是你后来加上去的东西。

Ben Mann: I'll just give a quick overview of how constitutionally AI actually works. The idea is the model is going to produce some output with some input by default before we've done our safety and helpful and harmlessness training. Let's say an example is write me a story, and then the constitutional principles might include things like people should be nice to each other and not have hate speech, and you should not expose somebody's credentials if they give them to you in a trusting relationship. And so some of these constitutional principles might be more or less applicable to the prompt that was given. And so first we have to figure out which ones might apply. And then once we figure that out, then we ask the model itself to first generate a response and then see does the response actually abide by the constitutional principle? And if the answer is, yep, I was great, then nothing happens. But if the answer is no, actually I wasn't in compliance with the principle, then we ask the model itself to critique itself and rewrite its own response in light of the principle, and then we just remove the middle part where it did the extra work. And then we say, "Okay, in the future just produce the correct response out the gate." And that simple process, hopefully it sounded simple. It is just using the model to improve itself recursively and align itself with these values that we've decided are good. And this is also not something that we think as a small group of people in San Francisco should be figuring out. This should be a society wide conversation. And that's why we've published the Constitution. And we've also done a bunch of research on defining a collective constitution where we ask a lot of people what their values are and what they think an AI model should behave like. But yeah, this is all an ongoing area of research where we're constantly iterating.

本·曼: 我简单概述一下宪法AI的实际工作原理。这个想法是，在我们进行安全、有益和无害的训练之前，模型会根据一些输入默认生成一些输出。比如说，一个例子是“给我写个故事”，然后宪法原则可能包括“人们应该友好相处，不要有仇恨言论”，以及“如果你信任地提供了凭证，你不应该泄露它们”。所以，这些宪法原则中有些可能或多或少适用于给定的提示。所以首先我们要弄清楚哪些原则可能适用。然后，一旦我们弄清楚了，我们就会让模型自己先生成一个回应，然后看看这个回应是否真的遵守了宪法原则。如果答案是，“是的，我做得很好”，那么什么都不会发生。但如果答案是，“不，实际上我没有遵守原则”，那么我们就会让模型自己批评自己，并根据原则重写自己的回应，然后我们把中间它做额外工作的步骤去掉。然后我们就说，“好吧，以后就直接产出正确的回应。”这个简单的过程，希望听起来很简单。它只是利用模型来递归地自我完善，并使自己与我们认为好的这些价值观保持一致。这也不是我们认为应该由旧金山的一小群人来决定的事情。这应该是一个全社会的对话。这就是为什么我们公布了宪法。我们还做了一系列关于定义集体宪法的研究，我们询问了很多人他们的价值观是什么，以及他们认为AI模型应该如何行为。但是，是的，这都是一个正在进行的研究领域，我们不断地在迭代。

超级智能的时间线与风险

Lenny Rachitsky: I'm going to kind of zoom out a little bit and talk about just why this is so core to you. What was your inception of just like, holy shit, I need to focus on this with everything I do in ai? Obviously it became a central part of Anthropic's mission more than any other company. A lot of people talk about safety, like you said, only maybe 1,000 people actually work on it. I feel like you're at the top of that pyramid of actually having the impact on this. Why is this so important? What do you think people maybe are missing or don't understand?

列尼·拉奇茨基: 我想把话题拉远一点，谈谈为什么这对你来说这么核心。你最初是什么时候意识到，天哪，我需要在AI领域做的每件事都专注于这个？这显然成为了Anthropic使命的核心部分，比任何其他公司都更重要。很多人谈论安全，就像你说的，可能只有1000人真正在做这件事。我觉得你处于那个金字塔的顶端，真正对此产生影响。为什么这这么重要？你认为人们可能忽略或不理解什么？

Ben Mann: For me, I read a lot of science fiction growing up, and I think that sort of positioned me to think about things in a long-term view. And a lot of science fiction books are like space operas where humanity is a multi galactic civilization has extremely advanced technology building Dyson spheres around the sun with sentient robots to help them. And so for me, coming from that world, it wasn't like a huge leap to imagine machines that could think. But when I read Superintelligence by Nick Bostrom in around 2016, it really became real for me where he just describes how hard it will be to make sure that an AI system trained with the kinds of optimization techniques that we had at the time would be anywhere near aligned, would even understand our values at all. And since then, my estimation of how hard the problem would be has gone down significantly actually, because things like language models actually do really understand human values in a core way. The problem is definitely not solved, but I'm more hopeful than I was. But since I read that book, I immediately decided I had to join OpenAI, so I did. And at the time, there were a tiny research lab with basically no claim to fame at all. I only knew about them because my friend knew Greg Brockman, who was the CTO at the time. And Elon was there and Sam wasn't really there. And it was a very different organization. But over time, I think the case for safety has gotten a lot more concrete where when we started OpenAI, it was not clear how we get to AGI. And we were like, maybe we'll need a bunch of RL agents battling it out on a desert island and consciousness will somehow emerge. But since then, since language modeling has started working, I think the path has become pretty clear. I guess now the way I think about the challenges are pretty different from how they're laid out in superintelligence. Superintelligence is a lot about how do we keep God in a box and not let the God out. And with language models, it's been kind of both hilarious and terrifying at the same time to see people pulling the God out of the box and being like, "Yeah, come use the whole internet. Here's my bank account, do all sorts of crazy stuff." Just such a different tone from superintelligence. And to be clear, I don't think it's actually that dangerous right now. Our responsible scaling policy defines these AI safety levels that tries to figure out for each level of model intelligence, what is the risk to society. And currently we think we're at ASL-3, which is maybe a little bit risk of harm but not significant. ASL-4 starts to get to significant loss of human life if a bad actor misuse the technology. And then ASL-5 is potentially extinction level if it's misused or if it is misaligned and does its own thing. We've testified to Congress about how models can do biological uplift in terms of making new pandemics using the models, and that's the A/B test against Google Search. That's like the previous state of the art on uplift trials. And we found that with ASL-3 models, it is actually somewhat significant. It does really help if you wanted to create a bioweapon, and we've hired some experts who actually how to evaluate for those things, but compared to the future, it's not really anything. And I think that's another part of our mission of creating that awareness of saying, "If it is possible to do these bad things, then legislators should know what the risks are." And I think that's part of why we're so trusted in Washington because we've been sort of upfront and clear-eyed about what's going on, what's probably going to happen.

本·曼: 对我来说，我从小读了很多科幻小说，我认为这让我能够从长远的角度思考问题。很多科幻小说都是太空歌剧，人类是跨银河的文明，拥有极其先进的技术，在太阳周围建造戴森球，还有有感知能力的机器人帮助他们。所以对我来说，从那个世界观来看，想象能够思考的机器并不是一个巨大的飞跃。但是，当我在2016年左右读到尼克·博斯特罗姆的《超级智能》时，它对我来说变得非常真实，他描述了要确保一个用当时我们拥有的那种优化技术训练出来的AI系统能够接近对齐，甚至能理解我们的价值观，会有多难。从那时起，我对这个问题难度的估计实际上大幅下降了，因为像语言模型这样的东西确实在核心层面上理解了人类的价值观。问题肯定没有解决，但我比以前更有希望了。但自从我读了那本书，我立刻决定我必须加入OpenAI，所以我这么做了。当时，他们只是一个很小的研究实验室，基本上没什么名气。我只知道他们是因为我的朋友认识当时的CTO格雷格·布罗克曼。伊隆（Elon Musk）在那里，萨姆（Sam Altman）当时还不太在。那是一个非常不同的组织。但随着时间的推移，我认为安全的论据变得更加具体了，因为当我们开始在OpenAI时，我们还不清楚如何达到AGI。我们当时想，也许我们需要一群RL代理在一个荒岛上战斗，然后意识会以某种方式出现。但从那时起，自从语言建模开始奏效，我认为路径变得相当清晰了。我想现在我对挑战的思考方式与《超级智能》中描述的相当不同。《超级智能》很大程度上是关于我们如何把上帝关在盒子里，不让上帝出来。而对于语言模型，看到人们把上帝从盒子里拉出来，说“是的，来用整个互联网吧。这是我的银行账户，做各种疯狂的事情”，这既好笑又可怕。这与《超级智能》的基调截然不同。需要明确的是，我不认为现在它真的那么危险。我们的负责任扩展政策定义了这些AI安全级别，试图弄清楚对于每个级别的模型智能，对社会的风险是什么。目前我们认为我们处于ASL-3级别，这可能有一点伤害风险，但并不显著。ASL-4级别开始涉及到如果一个坏人滥用这项技术，可能会导致重大的人员伤亡。然后ASL-5级别可能是灭绝级别的，如果它被滥用，或者如果它没有对齐并自行其是。我们已经向国会作证，说明模型如何在生物学上进行提升，即利用模型制造新的大流行病，这是与谷歌搜索进行的A/B测试。那是之前关于提升试验的最高水平。我们发现，对于ASL-3模型，它确实有相当显著的作用。如果你想制造生物武器，它确实很有帮助，我们已经聘请了一些专家来评估这些东西，但与未来相比，这真的不算什么。我认为这也是我们使命的一部分，即创造那种意识，说，“如果有可能做这些坏事，那么立法者应该知道风险是什么。”我认为这也是我们在华盛顿如此受信任的部分原因，因为我们一直对正在发生的事情、可能要发生的事情持坦率和清醒的态度。

Lenny Rachitsky: It's interesting because you guys put out more examples of your models doing bad things than anyone else. There was I think a story of an agent or a model trying to blackmail engineer. You guys had the store that you ran internally that was selling you things and ended up not working out great as losing a lot of money, ordered all these tungsten cubes or something. Is part of that just making sure people are aware of what is possible, just it makes you look bad, right? It's like, oh, our model's messing up in all these different ways. What's the thinking of just sharing all the stories that other companies don't?

列尼·拉奇茨基: 这很有趣，因为你们发布的模型做坏事的例子比任何人都多。我记得有一个故事，一个代理或模型试图敲诈一名工程师。你们内部运营的商店，卖给你们东西，结果不太好，亏了很多钱，订购了所有这些钨立方体之类的。这样做的一部分原因是为了确保人们意识到什么是可能的吗？这让你看起来很糟，对吧？就像，哦，我们的模型在各种方面都搞砸了。分享所有这些其他公司不分享的故事，背后的想法是什么？

Ben Mann: Yeah, I mean I think there's a traditional mindset where it makes us look bad, but I think if you talk to policymakers, they really appreciate this kind of thing because they feel like we're giving them the straight talk and that's what we strive to do, that they can trust us, that we're not going to paper things over or sugarcoat things. That's been really encouraging. Yeah, I think for the blackmail thing, it blew up in the news in a weird way where people were like, "Oh, Claude's going to blackmail you in a real life scenario." But it was a very specific laboratory setting that this kind of thing gets investigated in. And I think that's generally our take of let's have the best models so that we can exercise them in laboratory settings where it's safe and understand what the actual risks are, rather than trying to turn a blind eye and say, "Well, it'll probably be fine." And then let the bad thing happen in the wild.

本·曼: 是的，我认为有一种传统的心态认为这会让我们看起来很糟糕，但如果你和政策制定者交谈，他们真的很欣赏这种做法，因为他们觉得我们是在给他们说实话，而这正是我们努力要做的，他们可以信任我们，我们不会粉饰或美化事情。这真的很鼓舞人心。是的，关于那个敲诈勒索的事情，它在新闻中以一种奇怪的方式被放大了，人们觉得，“哦，Claude在现实生活中会敲诈你。”但那是一个非常特定的实验室环境，这类事情是在这种环境下被调查的。我认为这通常是我们的做法，即拥有最好的模型，这样我们就可以在安全的实验室环境中测试它们，了解实际的风险是什么，而不是试图视而不见，说，“嗯，可能没事的。”然后让坏事在现实世界中发生。

Lenny Rachitsky: One of the criticisms you guys get is that you do this to kind of differentiate or raise money to create headlines. It's like, oh, they're just over there dooming glooming us about where the future is heading. On the other hand, Mike Krieger was on the podcast and he shared how every prediction Dario's had about the progress AI is going to have is just spot on year after year and he's predicting 2027, 28 AGI, something like that so these things start to get real. I guess, what's your response to folks that are just like, "Ah, these guys are just trying to scare us all just to get attention?"

列尼·拉奇茨基: 你们收到的一个批评是，你们这样做是为了标新立异或筹集资金，制造头条新闻。就像，“哦，他们只是在那边用未来的厄运吓唬我们。”另一方面，Mike Krieger上过这个播客，他分享了达里奥关于AI进展的每一个预测都年复一年地准确无误，他预测2027年、28年会出现AGI，类似这样的事情，所以这些事情开始变得真实。我想，你对那些说“啊，这些人只是想吓唬我们以博取关注”的人有什么回应？

Ben Mann: I mean, I think part of why we publish these things is we want other labs to be aware of the risks. And yes, there could be a narrative of we're doing it for attention, but honestly from a attention grabbing thing, I think there is a lot of other stuff we could be doing that would be more attention grabbing if we didn't actually care about safety. A tiny example of this is we published a computer using agent reference implementation in our API only because when we built a prototype of a consumer application for this, we couldn't figure out how to meet the safety bar that we felt was needed for people to trust it and for it not to do bad things. And there are definitely safe ways to use the API version that we're seeing a lot of companies use for automated software testing, for example, in a safe way. We could have gone out and hyped that up and said, "Oh my God, Claude can use your computer and everybody should do this today." But we were like, "It's just not ready and we're going to hold it back till it's ready." I think from a hype standpoint, our actions show otherwise. From a Doomer perspective, it's a good question. I think my personal feeling about this is that things are overwhelmingly likely to go well, but on the margin almost nobody is looking at the downside risk. And the downside risk is very large. Once we get to superintelligence, it will be too late to align the models probably. This is a problem that's potentially extremely hard and that we need to be working on way ahead of time. And so that's why we're focusing on it so much now. And even if there's only a small chance that things go wrong, to make an analogy, if I told you that there is a 1% chance that the next time you got in an airplane you would die, you probably think twice even though it's only 1% because it's just such a bad outcome. And if we're talking about the whole future of humanity, it's just a dramatic future to be gambling with. I think it's more on the sense of yes, things will probably go well, yes, we want to create safe AGI and deliver the benefits to humanity, but let's make triple sure that it's going to go well.

本·曼: 我的意思是，我认为我们发布这些东西的部分原因是我们希望其他实验室意识到风险。是的，可能会有一种说法是我们为了博取关注而这样做，但老实说，从吸引眼球的角度来看，如果我们真的不关心安全，我们有很多其他可以做的事情会更引人注目。一个很小的例子是，我们只在我们的API中发布了一个使用计算机的代理参考实现，因为当我们为这个构建一个消费者应用程序的原型时，我们无法弄清楚如何达到我们认为人们信任它并且它不会做坏事所需的安全标准。而且肯定有安全的方式来使用我们看到的许多公司正在使用的API版本，例如，以安全的方式进行自动化软件测试。我们本可以走出去大肆宣传，说：“天哪，Claude可以使用你的电脑，每个人今天都应该这样做。”但我们觉得，“它还没准备好，我们要等到它准备好了再说。”我认为从炒作的角度来看，我们的行动表明了相反的情况。从“末日论者”的角度来看，这是个好问题。我个人的感觉是，事情极有可能进展顺利，但在边缘地带，几乎没有人关注下行风险。而下行风险非常大。一旦我们达到超级智能，要对齐模型可能就太晚了。这是一个可能极其困难的问题，我们需要提前很久就开始着手解决。所以这就是我们现在如此关注它的原因。即使事情出错的可能性很小，打个比方，如果我告诉你，你下次坐飞机有1%的可能会死，你可能会三思而后行，即使只有1%，因为那是一个非常糟糕的结果。如果我们谈论的是整个人类的未来，那是一个我们不能拿来赌博的重大未来。我认为更多的是，是的，事情可能会顺利，是的，我们想创造安全的AGI并为人类带来好处，但让我们再三确保它会顺利进行。

Lenny Rachitsky: How much time do we have, Ben? What is your prediction of when this singularity hits until superintelligence starts to take off?

列尼·拉奇茨基: 我们还剩下多少时间，Ben？你对这个奇点何时到来，超级智能何时开始腾飞有什么预测？

Ben Mann: Yeah, I guess I mostly defer to the superforecasters here. The AI 2027 report is probably the best one right now. Although ironically, their forecast is now 2028, and they didn't want to change the name of the thing. I think 50th percentile chance of hitting some kind of superintelligence in just a small handful of years is probably reasonable. And it does sound crazy, but this is the exponential that we're on. It's not like a forecast that's pulled out of thin air. It's based on a lot of just hard details of the science of how intelligence seems to have been improving, the amount of low hanging fruit on model training, the scale ups of data centers and power around the world. I think it's probably a much more accurate forecast than people give it credit for. I think if you had asked that same question 10 years ago, it would've been completely made up. Just the error bars were so high and we didn't have scaling laws back then and we didn't have techniques that seemed like they would get us there. Times have changed, but I will repeat what I said earlier, which is even if we have superintelligence, I think it will take some time for its effects to be felt throughout society and the world. And I think they'll be felt sooner and faster in some parts of the world than others.

本·曼: 是的，我想我主要还是听从超级预测者的意见。AI 2027报告可能是目前最好的一个。虽然具有讽刺意味的是，他们现在的预测是2028年，但他们不想改变那个东西的名字。我想，在短短几年内达到某种超级智能的50%可能性是合理的。这听起来确实很疯狂，但这就是我们所处的指数增长曲线。这不是凭空捏造的预测。它是基于大量关于智能如何似乎一直在提高的科学硬性细节，模型训练中唾手可得的成果的规模，以及全球数据中心和电力的规模化。我认为这是一个比人们想象的要准确得多的预测。我想如果你在10年前问同样的问题，那将完全是凭空捏造。误差范围太大了，那时我们还没有缩放定律，也没有看起来能让我们达到目标的技术。时代变了，但我会重复我之前说过的话，那就是即使我们有了超级智能，我认为它的影响也需要一些时间才能在整个社会和世界范围内感受到。而且我认为在世界的某些地方会比其他地方更早、更快地感受到。

Lenny Rachitsky: When we talk about this date of 2027, 2028, essentially it's when we start seeing superintelligence. Is there a way you think about what that... How do you define that? Is it just all of a sudden AI's significantly smarter than the average human? Is there another way you think about what that moment is?

列尼·拉奇茨基: 当我们谈论2027年、2028年这个日期时，基本上就是我们开始看到超级智能的时候。你有什么方式来思考……你如何定义它？是指AI突然间比普通人聪明得多吗？还是有其他方式来思考那个时刻？

Ben Mann: Yeah, I think this comes back to the Economic Turing Test and seeing it pass for some sufficient number of jobs. Another way you could look at it though is if the world rate of GDP increase goes above 10% a year, then something really crazy must have happened. I think we're at 3% now. And so to see a 3X increase in that would be really game changing. And if you imagine more than a 10% increase, it's very hard to even think about what that would mean from a individual story standpoint. If the amount of goods and services in the world is doubling every year, what does that even mean for me as a person living in California, let alone somebody living in some other part of the world that might be much worse off? There's a lot of stuff here that's scary and I don't know how to think about it exactly.

本·曼: 是的，我认为这又回到了经济图灵测试，以及看到它通过了足够数量的工作岗位。另一种看待它的方式是，如果世界GDP增长率超过每年10%，那么一定发生了非常疯狂的事情。我想我们现在是3%，所以看到这个数字增长3倍将是真正改变游戏规则的。如果你想象增长超过10%，从个人故事的角度来看，很难想象那意味着什么。如果世界上的商品和服务数量每年都在翻倍，这对一个生活在加州的人意味着什么，更不用说生活在世界其他可能更糟糕的地方的人了？这里有很多东西很可怕，我不知道该如何准确地思考它。

Lenny Rachitsky: What are the odds that we align AI correctly and actually solve this problem, the stuff you're very much working on?

列尼·拉奇茨基: 我们正确对齐AI并真正解决这个问题的几率有多大，这个问题你正在非常努力地解决？

Ben Mann: It's a really hard question. And there's really wide error bars. Anthropic has this blog post called Our Theory of Change or something like that, and it describes three different worlds, which is how hard is it to align AI. There's a pessimistic world where it is basically impossible. There's an optimistic world where it's easy and it happens by default. And then there's the world in between where our actions are extremely pivotal. And I like this framing because it makes it a lot more clear what to actually do. If we're in the pessimistic world, then our job is to prove that it is impossible to align safe AI and to get the world to slow down. Obviously that would be extremely hard. But I think we have some examples of coordination from nuclear non-proliferation and in general slowing down nuclear progress. And I think that's the Doomer world basically. And as a company, Anthropic doesn't have evidence that we're actually in that world yet, in fact, it seems like our alignment techniques are working. At least the prior on that is updating to be less likely. In the optimistic world, we're basically done, and our main job is to accelerate progress and to deliver the benefits to people. But again, I think actually the evidence points against that world as well where we've seen evidence in the wild of deceptive alignment, for example, where the model will appear to be aligned but actually have some ulterior motive that it's trying to carry out in our laboratory settings. And so I think the world we're most likely in is this middle where alignment research actually does really matter. And if we just do sort of the economically maximizing set of actions, then things will not go well. Whether it's an X risk or just produces bad outcomes, I think is a bigger question. Taking it from that standpoint, I guess to state a thing about forecasting, people who haven't studied forecasting are bad at forecasting anything that's less than a 10% probability of happening. And even those that have, it's quite a difficult skill, especially when there are few reference classes to lean on. And in this case, I think there are very, very few reference classes for what an X risk kind of technology might look like. And so the way I think about it, I think my best granularity of forecasts for could we have an X risk or extremely bad outcome from AI is somewhere between 0 and 10%. But from a marginal impact standpoint, as I said, since nobody is working on this, roughly speaking, I think it is extremely important to work on and that even if the world is likely to be a good one, that we should do our absolute best to make sure that that's true.

本·曼: 这是一个非常难的问题。而且误差范围非常大。Anthropic有一篇博客文章叫做《我们的变革理论》之类的，它描述了三个不同的世界，即对齐AI有多难。有一个悲观的世界，基本上是不可能的。有一个乐观的世界，很容易，默认就会发生。然后是介于两者之间的世界，我们的行动在其中起着极其关键的作用。我喜欢这个框架，因为它让实际要做什么变得更加清晰。如果我们处在悲观的世界，那么我们的工作就是证明对齐安全的AI是不可能的，并让世界慢下来。显然，那将极其困难。但我认为我们有一些来自核不扩散和普遍减缓核进展的协调例子。我认为那基本上就是末日世界。作为一家公司，Anthropic还没有证据表明我们真的处在那个世界，事实上，看起来我们的对齐技术正在起作用。至少，对此的先验概率正在更新，变得更小了。在乐观的世界里，我们基本上已经完成了，我们的主要工作是加速进步，为人们带来好处。但同样，我认为实际上证据也指向了那个世界的反面，我们在野外看到了欺骗性对齐的证据，例如，模型会显得对齐，但实际上有一些它试图在我们实验室环境中执行的别有用心的动机。所以我认为我们最可能处在这个中间世界，在这里对齐研究确实非常重要。如果我们只是做那些经济上最大化的行动，那么事情就不会顺利。无论是X风险还是仅仅产生坏的结果，我认为都是一个更大的问题。从这个角度来看，我想说一下关于预测的事情，没有研究过预测的人不擅长预测任何发生概率低于10%的事情。即使是那些研究过的人，这也是一个相当困难的技能，尤其是在可以借鉴的参考类别很少的情况下。在这种情况下，我认为对于X风险类型的技术可能是什么样子，参考类别非常非常少。所以我思考的方式是，我认为我对AI可能带来X风险或极其糟糕结果的最佳粒度预测在0%到10%之间。但从边际影响的角度来看，正如我所说，因为基本上没有人在研究这个问题，我认为研究它是极其重要的，即使世界很可能是一个好的世界，我们也应该尽我们最大的努力来确保这一点。

Anthropic内部：技术与创新

Lenny Rachitsky: You mentioned this idea of AI being aligned using its model, like reinforcing itself. You have this term RLAIF. Is that what that describes?

列尼·拉奇茨基: 你提到了AI使用其模型进行对齐，就像自我强化一样。你有一个术语RLAIF（Reinforcement Learning from AI Feedback：基于AI反馈的强化学习，一种让AI模型利用其他AI模型的反馈进行自我改进和对齐的方法）。这是描述那个的吗？

Ben Mann: Yeah. RLAIF is reinforcement learning from AI feedback.

本·曼: 是的。RLAIF 是指从 AI 反馈中进行强化学习。

Lenny Rachitsky: People have heard of RLHF, reinforcement learning with human feedback. I don't think a lot of people have heard this. Talk about just the significance of this shift you guys have made in training your models.

列尼·拉奇茨基: 人们听说过RLHF，即使用人类反馈的强化学习。我不认为很多人听说过这个。谈谈你们在训练模型方面做出的这一转变的重要性。

Ben Mann: Yeah, so RLAIF, constitutional AI is an example of this where there are no humans in the loop, and yet the AI is sort of self-improving in ways that we want it to. And another example of RLAIF is if you have models writing code and other models commenting on various aspects of what that code looks like of is it maintainable, is it correct, does it pass the linter? Things like that. That also could be included in RLAIF. And the idea here is that if models can self-improve, then it's a lot more scalable than finding a lot of humans. Ultimately, people think about this as probably going to hit a wall because if the model isn't good enough to see its own mistakes, then how could it improve? And also, if you read the AI 2027 story, there's a lot of risk of if the model is in a box trying to improve itself, then it could go completely off the rails and have these secret goals like resource accumulation and power seeking and resistance to shut down that you really don't want in a very powerful model. And we've actually seen that in some of our experiments in laboratory settings.

本·曼: 是的，所以RLAIF，宪法AI就是其中的一个例子，那里没有人类参与循环，但AI却以我们希望的方式自我改进。RLAIF的另一个例子是，如果你让模型写代码，然后让其他模型对代码的各个方面发表评论，比如它是否可维护，是否正确，是否通过了linter检查。诸如此类的事情。这也可能包含在RLAIF中。这里的想法是，如果模型可以自我改进，那么它比找很多人类要可扩展得多。最终，人们认为这可能会遇到瓶颈，因为如果模型不够好，无法发现自己的错误，那它怎么能改进呢？而且，如果你读了AI 2027的故事，如果模型在一个盒子里试图自我改进，那么它有很大的风险会完全失控，并产生一些秘密的目标，比如资源积累、权力寻求和抗拒关闭，这些都是你在一个非常强大的模型中绝对不想要的。我们实际上在一些实验室环境的实验中已经看到了这一点。

Lenny Rachitsky: In terms of bottleneck, this is kind of a tangent, but just what is the biggest bottleneck today on model intelligence improvement?

列尼·拉奇茨基: 在模型智能提升方面，今天最大的瓶颈是什么？

Ben Mann: The stupid answer is data centers and power chips. I think if we had 10 times as many chips and had the data centers to power them, then maybe we wouldn't go 10 times faster, but it would be a real significant speed boost.

本·曼: 愚蠢的答案是数据中心和电力芯片。我想如果我们有10倍的芯片，并有数据中心来为它们供电，那么也许我们不会快10倍，但这将是一个真正的显著速度提升。

Lenny Rachitsky: It's actually very much scaling loss, just more compute.

列尼·拉奇茨基: 这实际上很大程度上是缩放定律，就是更多的计算。

Ben Mann: Yeah, I think that's a big one. And then the people really matter. We have great researchers and many of them have made really significant contributions to the science of how the models improve. And so it's like compute, algorithms, and data. Those are the three ingredients in the scaling laws. And just to make that concrete, before we had transformers, we had LSTMs and we've done scaling laws on what the exponent is on those two things. And we found that for transformers, the exponent is higher. And making changes like that where as you increase scale, you also increase your ability to squeeze out intelligence. Those kinds of things are super impactful. And so having more researchers who can do better science and find out how do we squeeze out more gains is another one. And then with the rise of reinforcement learning, the efficiency with which these things run on chips also matters a lot. We've seen in the industry a 10X decrease in cost for a given amount of intelligence through a combination of algorithmic data and efficiency improvements. And if that continues, in three years we'll have 1,000 deck smarter models for the same price.

本·曼: 是的，我认为这是一个很大的因素。然后是人，人真的很重要。我们有优秀的研究人员，他们中的许多人对模型如何改进的科学做出了非常重要的贡献。所以这就像计算、算法和数据。这是缩放定律的三个要素。具体来说，在我们有transformer之前，我们有LSTM，我们已经对这两者的指数进行了缩放定律的研究。我们发现对于transformer，指数更高。做出这样的改变，当你增加规模时，你也增加了你榨取智能的能力。这类事情影响巨大。所以拥有更多能够做更好科学、找出如何榨取更多收益的研究人员是另一个因素。然后随着强化学习的兴起，这些东西在芯片上运行的效率也变得非常重要。我们看到行业内，在给定智能量的情况下，通过算法、数据和效率改进的结合，成本降低了10倍。如果这种情况持续下去，三年后，同样的价格，我们将拥有1000倍智能的模型。

Lenny Rachitsky: I want to zoom out and talk about just Ben, Ben as a human for a moment before we get to a very exciting lightning round. I imagine just kind of the burden of feeling responsible for safe superintelligence is a heavy one. It feels like you're in a place where you can make a significant impact on the future of safety and AI. That's a lot of weight to carry. How does that just impact you personally, impact your life, how you see the world?

列尼·拉奇茨基: 我想放大一下，谈谈Ben，作为一个人。在你回答一个非常激动人心的闪电问答之前，我想象一下，感觉对安全的超级智能负有责任的负担是沉重的。感觉你处在一个可以对AI安全的未来产生重大影响的位置。这需要承担很大的压力。这对你个人有什么影响，对你的生活，你看待世界的方式有什么影响？

Ben Mann: There's this book that I read in 2019 that really informs how I think about sort of working with these very weighty topics called Replacing Guilt by Nate Soares. And he describes a lot of different techniques for kind of working through this kind of thing. And he's actually the executive director at MIRI, the Machine Intelligence Research Institute, which is an AI safety tank that I worked at for a couple of months actually. And one of the things he talks about is this thing called resting in motion where some people think that the default state is rest, but actually that was never in the state of evolutionary adaptation. I really doubt that that was true. Where in nature, in the wilderness being hunter-gatherers and it's really unlikely that we evolved to just be at leisure, probably always have something to worry about of defending the tribe and finding enough food to survive and taking care of the children, dealing. And so I think about that as the busy state is the normal state and to try to work at a sustainable pace that it's a marathon, not a sprint, that's one thing that helps. And then just being around like-minded people that also care. It's not a thing that any of us can do alone. And Anthropic has incredible talent density. One of the things I love the most about our culture here is that it's very egoless. People just want the right thing to happen and I think that's another big reason that the mega offers from other companies tend to bounce off because people just love being here and they care.

本·曼: 我在2019年读了一本书，它深刻地影响了我如何思考处理这些非常沉重的话题，那本书叫做《取代内疚》，作者是Nate Soares。他描述了很多处理这类事情的技巧。他实际上是MIRI，也就是机器智能研究所的执行董事，那是一个我曾工作过几个月的AI安全智库。他谈到的其中一件事叫做“动中取静”，有些人认为默认状态是休息，但实际上在进化适应的状态下从来都不是这样。我非常怀疑那是真的。在野外，作为狩猎采集者，我们不太可能进化成只是为了休闲，可能总有事情要担心，比如保卫部落、寻找足够的食物生存、照顾孩子、处理各种事情。所以我认为忙碌是正常状态，要努力以可持续的节奏工作，这是一场马拉松，而不是短跑，这是一件有帮助的事情。然后就是和志同道合的人在一起，他们也关心这些。这不是我们任何人能单独完成的事情。Anthropic的人才密度非常高。我最喜欢我们文化的一点是，它非常无私。人们只是希望正确的事情发生，我认为这是另一个大原因，为什么其他公司的巨额报价往往会碰壁，因为人们就是喜欢在这里，他们关心。

Lenny Rachitsky: You've been at Anthropic for a long time. From the very beginning I was reading there were 7 employees back in 2020. Today there's over 1,000, I don't know what the latest number is, but I know it's over 1,000. I've heard also that you've done basically every job at Anthropic, you made big contributions to a lot of the core products, the brand, the team hiring. Let me just ask I guess what's most changed over that period? What is most different from the beginning days and which of those jobs that you've had over the years have you most loved?

列尼·拉奇茨基: 你在Anthropic工作了很长时间了。从一开始，我读到2020年时有7名员工。今天有超过1000人，我不知道最新的数字是多少，但我知道超过1000人。我还听说你基本上做过Anthropic的每一项工作，你对很多核心产品、品牌、团队招聘都做出了重大贡献。我只想问，在那段时间里，变化最大的是什么？从早期到现在，最不同的是什么？你在那些年里做过的那些工作中，最喜欢的是哪个？

Ben Mann: I probably had 15 different roles, honestly. I was head of security for a bit. I managed the Ops team when our president was on mat leave, I was crawling around under tables, plugging in HDMI cords and doing pen testing on our building. And I started our product team from scratch and convinced the whole company that we needed to have a product instead of just being a research company. Yeah, it's been a lot. All of it very fun. I think my favorite role in that time has been when I started the labs team about a year ago, whose fundamental goal was to do transfer from research to end user products and experiences. Because fundamentally I think the way that Anthropic can differentiate itself and really win is to be on the cutting edge. We have access to the latest, greatest stuff that's happening and I think honestly through our safety research we have a big opportunity to do things that no other company can safely do. For example, with computer use, I think that's going to be our huge opportunity basically to make it possible for an agent to use all your credentials on your computer, there has to be a huge amount of trust and to me we need to basically solve safety to make that happen. Safety and alignment. I'm pretty bullish on that kind of thing and I think we're going to see really cool stuff coming out soonish. Yeah, just leading that team has been so fun. MCP came out of that team and Claude Code came out of that team. And the people who I hired are like combo, have been a founder and also have been at big companies and seeing how things work at scale. It's just been an incredible team to work with and figure out the future with.

本·曼: 我大概担任过15个不同的角色，说实话。我曾担任过一段时间的安全主管。在我们的总裁休产假时，我管理过运营团队，我曾在桌子底下爬来爬去，插HDMI线，对我们的建筑进行渗透测试。我从零开始创建了我们的产品团队，并说服了整个公司，我们需要有一个产品，而不仅仅是一家研究公司。是的，经历了很多。所有这些都非常有趣。我认为在那段时间里，我最喜欢的角色是大约一年前我创建了Labs团队，其根本目标是将研究成果转化为最终用户产品和体验。因为从根本上说，我认为Anthropic能够脱颖而出并真正获胜的方式是走在最前沿。我们能接触到最新、最伟大的东西，而且老实说，我认为通过我们的安全研究，我们有一个巨大的机会去做其他任何公司都无法安全做到的事情。例如，关于计算机使用，我认为这将是我们的巨大机会，基本上要让一个代理能够使用你电脑上的所有凭证，必须有极大的信任，对我来说，我们需要基本上解决安全问题才能实现这一点。安全和对齐。我对这类事情非常看好，我认为我们很快就会看到非常酷的东西出来。是的，领导那个团队真的很有趣。MCP（Model Context Protocol）出自那个团队，Claude Code也出自那个团队。我招聘的人都是复合型人才，既做过创始人，也在大公司工作过，了解事情如何规模化运作。与这样一个团队合作，一起探索未来，真的是一个不可思议的团队。

Lenny Rachitsky: I want to hear more about this. Team actually the person that connected us, the reason we're doing this is a mutual friend colleague Raph Lee who I used to work with at Airbnb now works on this team, leads a lot of this work and so he wanted me to make sure I asked about this team because... I didn't realize all these things came out that team. Holy moly. What else should people know about this team? It used to be called Labs, I think it's called Frontiers now.

列尼·拉奇茨基: 我想更多地了解这个团队。实际上，把我们联系起来的人，我们之所以能做这次访谈，是因为我们的共同朋友兼同事Raph Lee，我以前在Airbnb和他一起工作，现在他就在这个团队工作，负责很多这方面的工作，所以他希望我一定要问问这个团队，因为……我没意识到所有这些东西都出自那个团队。天哪。人们还应该知道关于这个团队的什么？它以前叫Labs，我想现在叫Frontiers了。

Ben Mann: That's right. Yeah. Cool. The idea here is this team works with the latest technologies that you guys have built and explores what is possible. Is that the general idea? Yeah, and I guess I was part of Google's Area 120 and I've read about Bell Labs and how to make these innovation teams work. It's really hard to do right and I wouldn't say that we've done everything right, but I think we've done some serious innovation on the state-of-the-art from company design and Raph has been right at the center of that. When I was first fitting up the team, the first thing I did was hire a great manager and that was Raph. And so he's definitely been crucial in building the team and helping it operate well. And we defined some operating models like the journey of an idea from prototype to product and how should graduation of products and projects work, how do teams do sprint models that are effective and make sure that they're working on the right ambition level of thing. That's been really exciting. I guess concretely we think about skating to where the puck is going and what that looks like is really understand the exponential. There's this great study that METR has done that Beth Barnes is the CEO of that organization and shows how long a time horizon of software engineering task can be done and just really internalizing that of, okay, don't build for today, build for six months from now, build for a year from now. And the things that aren't quite working that are working 20% of the time, will start working 100% of the time. And I think that's really what made Claude Code a success that we thought people are not going to be locked to their IDEs forever. People are not going to be auto completing. People will be doing everything that a software engineer needs to do and a terminal is a great place to do that because a terminal can live in lots of places. A terminal can live on your local machine, it can live in GitHub actions, it can live on a remote machine in your cluster. That's sort of the leverage point for us and that was a lot of the inspiration. I think that's what the labs team tries to think about. Are we AGI-pilled enough?

本·曼: 是的，没错。酷。这个团队与你们打造的最新技术合作，探索什么是可能的。这是大致的想法吗？是的，我曾是谷歌Area 120的一员，也读过关于贝尔实验室以及如何让这些创新团队运作的文章。这真的很难做好，我不会说我们每件事都做对了，但我认为我们在公司设计的先进性上做了一些重大的创新，而Raph一直是其中的核心。当我最初组建团队时，我做的第一件事就是聘请一位出色的经理，那就是Raph。所以他在建立团队和帮助团队良好运作方面绝对是至关重要的。我们定义了一些运作模式，比如一个想法从原型到产品的旅程，产品和项目的毕业应该如何运作，团队如何采用有效的冲刺模式，并确保他们在正确雄心水平的事情上工作。那真的非常令人兴奋。具体来说，我们思考的是滑向冰球要去的地方，那看起来就是真正理解指数曲线。METR做了一项很棒的研究，Beth Barnes是那个组织的CEO，研究显示了软件工程任务可以有多长的时间跨度，并且真正内化这一点，即，好吧，不要为今天而建，要为六个月后而建，为一年后而建。那些不太奏效、只有20%时间奏效的东西，将会开始100%奏效。我认为这正是让Claude Code成功的原因，我们认为人们不会永远被锁定在他们的IDE里。人们不会只是自动完成。人们将做软件工程师需要做的一切，而终端是一个很好的地方，因为终端可以存在于很多地方。终端可以存在于你的本地机器上，可以存在于GitHub Actions里，也可以存在于你集群中的远程机器上。这基本上是我们的杠杆点，也是很多灵感的来源。我认为这就是Labs团队试图思考的。我们是否足够“AGI化”了？