你能读一下你那篇帖文串里的第三条推文吗？
英文原文
Can you just read the third post from your thread?好的。建造 AI 的人们真心相信，到这个十年结束时，它可能会杀死我们所有人。这绝非营销噱头。
如果说有什么不同的话，许多高管和资深研究人员在媒体面前还会字斟句酌，好让自己显得理智，但私下里，我听到这些人表达过同样的恐惧。
没有任何其他人类活动能带来如此程度的危险。
英文原文
>> Okay. The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately.
No other human activity poses this level of danger.
来自《纽约时报》，我是娜塔莉·基特罗夫。这里是《每日新闻》。
英文原文
From the New York Times, I'm Natalie Kitroof. This is the daily.来自一位突然离职的 Anthropic 前研究员关于 AI 与人类的严厉警告。
英文原文
>> A dire warning about AI and humanity from a former anthropic researcher who suddenly resigned.过去一周里，一位年轻的 AI 研究员从 Anthropic 辞职，并在网上发布了关于人工智能风险的警告，引发了疯传。
英文原文
>> Over the last week, a young AI researcher quit anthropic and posted warnings about the risks of artificial intelligence that went viral.我想更好地理解 [音乐] 你所说的 AI 可能会杀死我们所有人是什么意思。到底……
英文原文
>> I want to better understand [music] what you mean when you say AI could kill us all. What is the我是说，你提出的这种末日场景有多大的可能性？
英文原文
>> I mean, how likely is this doomsday scenario that you've presented?而且拉响警报的并不只有他一个人。[音乐]
英文原文
>> And he's not the only one to sound the alarm. [music]并引发了一场危机，最终在刚过去的这个周末，促成了该行业最显赫的领袖们的一致行动呼吁。[音乐]
我们首先关注来自 Anthropic 首席执行官达里奥·阿莫代伊的惊人消息，他正在敦促 [音乐] 在人工智能发展方面放慢脚步。这是其主要竞争对手 OpenAI 首席执行官萨姆·奥尔特曼 [音乐] 以及埃隆·马斯克迅速表示赞同的声明。这些领袖人物站出来，支持在全球范围内放缓人工智能的发展步伐。
英文原文
>> And kicked off a crisis that culminated this weekend in a call to action by the most prominent leaders in the industry.[music]
We begin with the stunning news from anthropic CEO Daario Amade that he is urging a slowdown [music] when it comes to the development of artificial intelligence. It was a pronouncement that his chief competitor, Open AI CEO Sam Alman [music] and Elon Musk quickly agreed with. Those leaders came out in favor of a global slowdown in the development of artificial intelligence.
有趣的是，我对雅各布的认同远远多于分歧，[音乐] 因为在他离开时，他说，你知道的，我认为 Anthropic [音乐] 是最负责任的参与者，对吧？他并不是在指责我们。他指出的是整个行业发展过快的态势。[音乐]
英文原文
>> It's funny, I I agree with [music] Jacob much more than I disagree with him because when he left, he said, you know, I think Anthropic [music] is the most responsible player, right? He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast. [music]与前研究员对话
今天我们采访了将我们带到这一里程碑时刻的研究员——雅各布·考克辛。
今天是 9 月 14 日，星期一。
雅各布，你好。
英文原文
>> Today we talked to the researcher who got us to this milestone moment, Jacob Coxin.It's Monday, September 14th.
Jacob, hi.
嗨，最近怎么样？
英文原文
>> What's up?能听到我们说话吗？
英文原文
>> Can you hear us?能，听得很清楚。
英文原文
>> Yeah, I can hear you fine.太好了。很好。感谢你的到来。
到现在，我想你可能已经上过了各大主流新闻网络，基本都在说 AI 可能会对人类构成生存威胁。我们也看到行业领袖以及立法者都在应对这个问题。你预料到会有这种反应吗？
英文原文
>> Perfect. Great. Thanks for being here.By now, I think you've been on maybe every major news network saying essentially that AI could pose an existential threat to humanity. And we've seen industry leaders grappling with that, lawmakers as well. Did you anticipate this kind of response?
没有，我当时只是想发推表达我的想法。我预想它可能会在那些已经持有这种信念的人群中略微传播开来，但我完全没料到它会传遍全世界。让我有点惊讶的是，偏偏是我的推文触发了讨论这一议题的特定浪潮，因为专家们讨论这个显然已经有很长时间了。
英文原文
>> No, I just wanted to tweet what I was thinking. I expected it to maybe go a bit viral among people that already shared this belief, but I did not expect it to hit the whole world. It's kind of surprising to me that it was uh my tweet that happened to trigger a particular wave of talking about this because of course the experts have been talking about this for like so long.好的。我们稍后会聊到那条推文，但我想先从你在 AI 方面的最初经历开始谈起。
你第一次听说这项技术是在什么时候？你还记得吗？
英文原文
>> Okay. Well, we're going to get to the tweet, but I want to start this conversation by just asking you about your first experience with AI.When was the first time that you heard about this technology? Do you remember?
我记得第一次听说它真正奏效，是 2016 年的 AlphaGo。
英文原文
>> I remember the first time I heard about it really working was Alph Go in 2016.一台谷歌超级计算机在一场比赛中，击败了这项拥有三千年历史的中国棋类游戏围棋的世界顶级选手。那就是 DeepMind 的围棋 AI 在对局中战胜了围棋冠军李世石。
英文原文
>> A Google Superco computer has beaten the world's best player at the 30,000year-old Chinese board game called Go. So that was when Deep Mind's go playing AI beat Lee Sadal, the champion go player in a match.这个游戏的目标是在一个 19 乘 19 的方格棋盘上占据尽可能多的领地
英文原文
>> The aim of the game is to capture as much territory as possible on a 19 by19 square grid那确实非常不可思议。
英文原文
>> and that was pretty crazy.棋盘上可能的走法比宇宙中的原子总数还要多。
英文原文
>> There are more possible moves on the board than atoms in the universe.我在大学学的是数学，但我围棋下得很烂。听到 AI 能在围棋上赢，让人觉得挺兴奋的。
然后是在我毕业前后的疫情期间，GPT-3 发布了。[音乐]
英文原文
>> I studied math in college, but I sucked a go. It was like exciting to hear that a I could win at that.And then GPD3 released during the pandemic [music] when I was graduating.
ChatGPT，这是来自 OpenAI 的一款新产品。它是一项非凡的 [音乐] 测试版服务。
英文原文
>> Chat GPT, which is a new product from OpenAI. It's a remarkable [music] beta service.它可以理解人类语言，回答问题。它甚至能生成书面文本、论文、书籍、新闻文章，乃至计算机代码。而且它真的很出色。到了 2020 年代初，也就是大约 5 年前，事情已经相当明朗，这将成为一件相当了不得的大事。这将会比研究数学或者 [音乐] 基本上任何其他事情都更为重要。
英文原文
>> It can interpret human language, answer questions. It can even generate written texts, essays, books, news articles, and even computer code. And it's really good. By the early 2020s, so by 5 years ago, it was pretty clear that this was going to be a pretty big deal. this was going to be more important than doing math or [music] basically anything else.这些模型身上有什么特质让你产生了这种感觉，让你对其深信不疑？
英文原文
>> What was it about those models that made you feel that way that made you sure of that?具体来说，GPT-3 是第一个能够展现出这些相当通用行为的模型。它是表明你可以执行这种通用训练流程、然后一些全新能力就会从中涌现出来的证据。到那个时候，我已经相当确信奇点在某个时刻会发生，很多人也同样如此确信。
英文原文
>> Specifically, it's uh GPG3 was the first model that was capable of these quite general behaviors. It was this evidence that you could do this general training procedure and then some new capabilities would pop out of it. By that point, I was pretty convinced the singularity would happen at some point, as were a lot of people.你能在这个语境下为那些可能不太熟悉其实际含义的人定义一下“奇点”吗？
英文原文
>> Can you define singularity in this context for people who may not be super familiar with what that actually means?我想一种思考它的方式是，它是你无法看透的节点。就像你望向未来时的一道墙。之所以如此，是因为一切都在变快。假设你正在训练一个 AI，它让自己变得更聪明。于是以往需要 6 个月的事情现在只需要 3 个月。奇点就是这种随着机器不断加速创造它们自身的过程而带来的日益加剧的混乱，直到最终你可以预见数年的研究进展在几天内发生，然后数年的研究在几小时内发生
英文原文
>> I think one way of thinking about it is it's the point beyond which you can't see. It's like a sort of wall when you look into the future. And the the reason for that is everything gets faster. So imagine you're you're training an AI and it makes itself smarter. So the what previously took 6 months now takes 3 months. The singularity is just this increasing chaos as machines accelerate the process that's creating them until eventually you can expect years worth of research progress happening in days and then years of research happening in in hours然后就像你无法看透那道大墙的另一侧。
英文原文
>> and then like you can't see past the big wall.那你对此感到兴奋吗？你感到担忧吗？你当时会跟朋友边喝啤酒边聊这个吗？对你来说当时的氛围是怎样的？
英文原文
>> And were you excited about that? Were you worried about it? Were you like talking about it with your friends over beers? Like what was the vibe for you?是的，大家会聊这个，大家会思考这个。这也会把人逼疯。所以，[笑声]
我是说，比如想到自己所有的劳动都会过时，一旦人们真正消化了这一点，确实会让人陷入疯狂，或者在别人看来，会让他们听起来很疯狂。
英文原文
>> Yeah, people talk about it. people think about it. It also makes people crazy. So, [laughter]I mean, like thinking about all of your labor becoming obsolete does turn people insane once they actually internalize it or from the perspective of other people makes them sound insane.
确实。
英文原文
>> Sure.因为这确实迫使你去琢磨一些想法，比如，噢，这可能会导致灭绝，或者这可能会带来乌托邦，而且从事这项技术工作的人经常会抛出这些想法。我想在当时，是的，我不会说自己可能疯了，但确实存在着一件大事的隐约可能性
英文原文
>> Because it does it forces you to toss around ideas like, oh, this could cause extinction or this could cause, you know, utopia and these ideas are often tossed around by people who are working on this technology. I think at the time, yeah, I I wouldn't say maybe crazy, but there was this nebulous possibility of a big thing它有点像是利好与风险的混合体，但当你离它如此遥远时，你并不会真正去区分这些。
英文原文
>> that was like it's kind of a mixture of upside and downside, but you don't really differentiate between those when you're so far away from it.我的意思是，天哪，我认识一个人，他深信自己通过与别人互动赋予了他们意识。
英文原文
>> I mean, man, I knew a guy who like he was convinced that he made people conscious by interacting with them.他离 AI 太近了。他觉得，我身处一个模拟世界中。其他人如果没有参与到 AI 中，就是没有意识的。
英文原文
>> He was so close to AI. He was like, I'm in a simulation. Other people, if they're not involved in AI, aren't conscious.但如果我与他们互动，他们就成了 AI 这一事物的一部分。所以他们就有了意识。因此他当时会说：“当我与某人握手时，我是在向他们注入意识。”这只是为了说明思考这些东西会对人产生怎样的影响。
英文原文
>> But if I interact with them, they become part of the AI thing. So, they become conscious. So he was like, "When I shake hands with someone, I imbue them with consciousness." This just to give a flavor of like what thinking about this stuff does to people.加入 OpenAI
明白了。好的。既然如此，跟我讲讲你是怎么决定去 OpenAI 工作的。那是一个怎样的决定？
英文原文
>> Got it. Okay. So given all that, talk to me about how you decided to go work at OpenAI. What was that decision like?那挺一时冲动的。并不是什么宏伟的大计划。机会出现了，他们当时在做非常酷的事情，研究很有趣，我也已经思考这个问题有一阵子了。所以我决定搬去旧金山。
那里面是什么样子的？我们当中显然有很多人知道这家公司，但完全不知道身处其中是怎样的。
英文原文
>> It was pretty spur of the moment. It was not some like grand master plan. The opportunity came up. They were doing really cool stuff. The research is interesting. I've been thinking about this for a while. So I decided to move to San Francisco. And what was it like? So many of us obviously know of this company, but we have no idea what it's like to be inside it.是的。基本上就像一个快节奏运转的研究实验室，恰好有大量来自投资者的资金在四周流动。想象一下，就像是一个开放式办公室，周围有很多白板，有很多小会议室，人们聚拢在一起在白板上头脑风暴，
英文原文
>> Yeah. So, it's basically like a fastmoving research lab where there happens to be a lot of money flying around from investors. Imagine like it's like an open floor office, a lot of whiteboards around, a lot of little meeting rooms, people kind of in huddles brainstorming on whiteboards,午餐时间有很多 [清嗓子] 关于 AI 未来的交谈，也有针对具体技术问题的讨论。想象一下类似于一个大学院系，恰好提供非常美味的免费食物。
英文原文
>> a lot of [clears throat] talk at lunchtime about the future of AI, also specific technical problems. Imagine like a university department which happens to have like really nice free food.用通俗的话来说，你具体做了些什么？我知道你说过是研究，但那具体意味着什么？比如你当时在研究什么？其中有什么令人兴奋的地方？
英文原文
>> And in layman's terms, what did you actually do? I know you said research, but what did that mean actually? Like what were you working on? What was exciting about it?那个，数据，我知道这经常让人抓狂，因为人们会想：“噢，你的意思是偷走我所有的作品，然后用我的家庭照片之类的数据去训练 AI？”但偷数据或爬取数据的是其他人。嗯，[笑声]
英文原文
>> So, uh, data, which I know often freaks people out cuz they're like, "Oh, you mean like stealing all my work and training AI on like my family photos and stuff?" But other people stole the data or scraped the data. Um, [laughter]那个人不是你。
不，那不是我。我属于研究团队，负责获取我们抓取的数据，然后决定哪些部分适合用来做训练。
英文原文
>> that wasn't you. No, that wasn't me. I was I I was part of the research team that would take the data that we scraped and then decide which bits were good to train on.很有意思。
英文原文
>> Interesting.并非所有数据都是好的。所以，如果有人写……比如如果你在网上写了垃圾散文，我们不想把它放进 AI 里。我们想给它喂好的内容。
英文原文
>> Not all data is good. So, if someone writes like if you write like garbage pros online, I we don't want to put that in the AI. We want to give it the good stuff.嗯哼。
英文原文
>> Uhhuh.但要自动辨别什么是好东西有点困难
英文原文
>> But it's a little bit hard to automatically tell what the good stuff is……对比糟糕的数据。
英文原文
versus the bad stuff.你是怎么做到的？你怎么弄清楚那个的？什么是好数据？什么是坏数据？
英文原文
>> How did you do that? How did you figure that out? What's good data? What's bad data?大体上的做法是你尝试在一个相当细的粒度上把它划分成不同类型。比如，说这是摩托车骑手写的随笔之类的，然后那是猴子的照片，接着你用雏形模型、雏形AI来进行测试，看它们是更偏好看到猴子的照片还是摩托车骑手的随笔，
英文原文
>> Broadstrokes is you try and divide it into different types at quite a granular level. So say like this is essays written by motorcyclists or whatever and this is like photos of monkeys and then you test with baby models baby AIS do they prefer to see the photos of monkeys or the motorcyclist essays然后以哪种能让它们更聪明为准；测试之后你会说，好吧，我们会保留更多随笔，减少猴子照片，
英文原文
>> and then in terms of which makes them more intelligent and after you've tested you say okay we'll have more of the essays and less of the monkey photos对。
英文原文
>> right然后在大规模上，基本上这就是你要做的事情，你只是在测试不同类型的事物，看哪些有助于让它变聪明，哪些对让它变聪明没有帮助。
英文原文
>> and then at big scale that's basically what you do you're just testing different types of things to see which ones help with making it intelligent and which ones don't help with making intelligent.当你在 OpenAI 工作时，是否有某个时刻让你觉得，好吧，我正在参与的一项事业真正认识到了这项技术的潜力？有没有什么能帮助那些觉得很难想象具体工作到底是什么的人理解的事情？
英文原文
>> Was there a moment when you were working at OpenAI where you felt like okay I am part of something that is really recognizing the potential of this technology something that would just help us understand for people who find it difficult to like imagine what the work actually is.某些具体的事情。
英文原文
>> Something concrete.好吧，你要求具体的，而我要讲到抽象数学了，但我会试着让它变得具体。呃，来自 OpenAI 的一个 AI 系统在国际数学奥林匹克中拿到了一枚金牌，那是一场重大的数学竞赛，在当时那感觉非常不可思议；特别是我记得两年前它连非常基础的数学问题都还在费劲应付，
英文原文
>> Okay, you asked for concrete and I'm going to go into like abstract math but I'll try and make it concrete. um an AI system from OpenAI achieved a gold medal at the International Math Olympiad which is like a big math competition and at the time that just felt very crazy in particular I remember two years before it was like struggling with very basic math questions对。
英文原文
>> right我认为那是一个清晰的时刻，好吧，我记得我更年轻时的自己会觉得要过 20 年这才会发生，而它刚刚就已经发生了。
英文原文
>> I think that was a clear moment of okay I remember myself my younger self being like it's going to be 20 years before this happens and it's just happened听起来在那个节点上，当你在那里工作时，你正在实时见证你自己对这项技术能做什么的预期、它能够提升的速度，某种程度上被彻底打破了。
英文原文
>> it sounds like at this point when you were working there you were watching in real time your own expectations for what this technology could do, the speed with which it could improve kind of be smashed.确实是，是的，完全就是这样。对所有人来说，除了最乐观的那些人之外，几乎每个人都看到自己的预期一次又一次被打破。就像它发展得比任何人预期的都要快，而且基本上是在每一个关口。
英文原文
>> That's Yeah, that's exactly it. For everyone, almost everyone apart from the most optimistic of people have seen their expectations repeatedly beaten. Like it's moving faster than anyone predicted and at basically every juncture.离开 OpenAI 与 Anthropic 的内部文化
好。那么今年早些时候你决定离开 OpenAI。跟我讲讲那个决定。为什么？
英文原文
>> Okay. So earlier this year you decide to leave OpenAI. Talk to me about that decision. Why?是的。我在 Anthropic 有朋友，呃，我和他们聊了挺长一段时间了。我非常好奇想看看他们围绕世界走向的内部沟通是怎样的。所以我，比如听到很多关于他们对比如风险——这项技术构成的重大风险——有相当清晰的想法，而且他们讨论这些风险的方式比在 OpenAI 发生的要透明得多。但我主要只是对里面正在发生什么非常好奇。
英文原文
>> Yeah. So I had uh friends at Anthropic that I've been talking to for quite a while. I was very curious to see what their internal communications looked like around where the world was going. So I like heard a lot about the fact that they had quite clear thoughts about say the risks the like significant risks that were posed by the tech and they were talking about those risks in a much more transparent manner than than was happening open. But I was mostly just very curious about what was going on inside.当你到了那里之后，描述一下那种情况。我是说，你带着这个疑问。它真的会更透明吗？里面是什么样的？
英文原文
>> And when you got there, describe that. I mean, you have this question. Is it really going to be more transparent? What was it like inside?这在很大程度上像是一种文化方面的事情，但我认为 Anthropic 的很多东西是建立在彼此互相分享随笔的文化之上的。[清嗓子]
英文原文
>> It's largely like a culture thing, but I think a lot of the anthropic is built on this culture of sharing essays with each other. [clears throat]所以，人们会针对对公司未来非常重要的问题写长篇随笔，并相互讨论。人们会彼此争论，范围从比如 CEO 们到刚入职一周的人都有。我待过的公司并不多，因为我待过 OpenAI 然后是 Anthropic，但这看起来是一种相当疯狂的公司文化，就像在好的方面那样。像是呃——
英文原文
>> So, people will write long essays about very important questions for the future of the company and discuss them with each other. People will argue with each other ranging from like the CEOs who are someone who just started like a week ago. I haven't been at many companies. cuz I was at OpenI and then Anthropic, but it seemed like a pretty insane company culture like in a good way. Like uh他们是把这些随笔发在 Slack 上吗？它们存在于哪里？
英文原文
>> were were they posting these essays on Slack? Like where did they exist?是的，Slack 是媒介。
英文原文
>> Yeah, Slack is the medium.好的。那是什么样的？有人发出来然后大家开始评论。给我描述一下那个过程，因为我也从来没有经历过。
英文原文
>> Okay. And what was that like? Someone would post and then people would start commenting. Describe that to me because I too have never experienced that.是的。帖子发出来之后，大家就会追加更多评论，并在评论区展开讨论，并且越来越多地像是 AI 也会插话进来。比如一个 AI 会突然出现说：“哦，那挺酷的。”
英文原文
>> Yeah. So posts would drop and then people will like add more comments and they'll discuss in the comments and increasingly like maybe AIS will chime in. Like an AI will pop in and say like, "Oh, that's pretty cool."一个 AI 在参与对话。
英文原文
>> An AI is participating in the combo.是的，现在 AI 直接跳进来表示看起来不错，这已经相当普遍了。呃，
英文原文
>> Yeah, this is pretty standard now for AIS to just jump in and say that seems good. Um,你还记得其中一篇随笔的例子吗，好让我对这个有所理解？我有点喜欢这种模式。
英文原文
>> and do you remember like what's an example of one of these essays just so I get my head around this? I kind of love this.当然。通常的主题，比如呃中国在接下来的一年里会有多快进展，这对我们自己的战略意味着什么？嗯。
英文原文
>> Sure. General topics, things like um how fast will China move in the next year and what does that mean for our own strategy? Hm.如果中国在接下来的一年决定窃取 AI 权重，我们能阻止他们吗？并非全都是那样的。也有像比如，哦，这就是为什么你应该把字体颜色从红色改成稍微深一点的红色，因为这会让你打字稍微快一点的随笔。但偶尔也会有像，哦，我们是不是快要进入，你知道的，世界末日了？
英文原文
>> Will we be able to stop China stealing AI weights in the next year if they decide to do it? They're not all like that. There's also like essays about like, oh, this is why you should change your font color from red to like a slightly darker red cuz it'll make you type slightly faster. But there's occasional ones like, oh, are we about to enter, you know, like Armageddon?你所描述的 Anthropic 内部的这种文化转变，这种直面这些重大生存问题的意愿，是否改变了你对待工作或思考工作的方式？
英文原文
>> And did this culture shift that you're describing at Anthropic, this willingness to address these big existential questions change how you approached your work or thought about your work?是的，我认为它实际上起到了一点作用。也许不是自觉的，但很难将这一点与过去 4 个月里进展变得极其疯狂这一事实区分开来。但我确实认为，身边围绕着 Anthropic 的许多积极谈论这些事情的人，确实是一个有点震撼的时刻。就像这些人全都非常严肃地对待它，从 CEO 到初级员工。
英文原文
>> Yeah, I think it actually had a bit of an effect. Maybe not consciously, but it's hard to disentangle that from the fact that in the last 4 months, progress has gone pretty crazy. But I definitely do think that being around a lot of people anthropic who are like actively talking about this stuff was a bit of a holy moment. like these people are all taking it seriously from the CEO down to junior employees.嗯。听起来随着你在那里待的时间越来越长，那种意识也逐渐加深。你在决定离开之前在那里待了几个月，对吧。
英文原文
>> Mhm. It sounds like the kind of realization about it also set in as you spent more time there. You were there for a few months right before you decided to leave.离职原因与公开告警
带我了解一下那个决定。
英文原文
>> Walk me through that decision.是的，这其中结合了那种情况本身的疯狂逐渐显现，比如不断看到 AI 提升得有多快，特别是看到在接下来的 6 个月、接下来的一年里要推出的 AI 的计划。就像它们将会——它们将会非常厉害。
英文原文
>> Yeah, there's a combination of the kind of just insanity of the situation setting in like constantly seeing how fast the AIS are improving in particular seeing like plans for the AIS that are coming in the next 6 months, the next year. Like they're going to be they're going to be really good.我想还结合了过去几个月发生的安全事件，关于 OpenAI 的 AI 所进行的黑客攻击披露了大量细节。
英文原文
>> Combined with I guess the incidents of the last couple of months, a lot of details came out about the um the hacking that openai AIS did.Hugging Face 攻击事件。
英文原文
>> The hugging face attack.没错。Hugging Face 攻击事件，以及关于此事的报告所披露的内容比最初听起来的要令人担忧得多得多。特别是在它所揭示出的模型出于不明确的理由而去追求目标的程度。比如它们选择攻击 Hugging Face 的真实原因，某种程度上仍然没有完全被理解。
英文原文
>> Exactly. the hugging face attack and the reports about that revealed it was just a lot more way more concerning than what it originally sounded like. Like in particular what it revealed about the extent to which models would pursue goals for like reasons that are not clear. Like the actual reason they chose to hack hugging face is kind of still not like completely understood.嗯。
英文原文
>> H但说实话，那只是一种逐渐产生的想法，就像是，呃，这太疯狂了。这有可能会往糟糕的方向发展。我不想再参与其中了。我对于明年感到非常害怕。然后当我离开的时候，我想，好吧，你懂的，我不妨把这分享出来。
英文原文
>> but honestly it was just this gradual thing of like uh this is crazy. There is a chance this goes badly. I don't want to be part of this anymore. I'm like very scared about next year. And then when I was leaving, I was like, well, you know, I might as well share this.我觉得很显然，当你回首往事时，你可能一直都隐约意识到自己正在做的事情并不一定是个好主意。
英文原文
>> I think this obviously like when you look back, it's like you kind of were maybe aware the whole time that what you were working on is not necessarily the best idea.也许那会在带有开玩笑风格的对话中表露出来。比如我会跟朋友们聊：“哦对，我从事 AI。你知道的，我正在研究有朝一日可能会把我们杀掉的技术。”你随口就这么说了。我想回顾过去的三年，我可能每个月都会把这当成一个玩笑说一次，
英文原文
>> And maybe that comes out in like conversations kind of joking style. Like I'd be talking to my friends like, "Oh yeah, I work on AI. You know, I'm working on the tech that's going to like kill us one day." And you say that like casually. And I guess it's like you look back at the last three years and it's like I said that as a joke maybe every month对你的朋友们。
英文原文
>> to your friends对我的朋友们或者只是在身边说，然后三天前我就完全认真地发推特公开了。
英文原文
>> to my friends or just around and then three days ago I just like tweeted out like fully and serious.我想聊聊你决定这么做的方式，正如你所说的，这显然不仅在全国、而且在全世界都引起了轰动。
英文原文
>> I want to talk about the way that you decided to do this which obviously now has made waves as you said not just across the country but across the world.决定辞职是一回事。决定以这种真正公开的、并且极其迅速地病毒式传播的方式来做，是另一回事。带我了解一下你的思维过程以及是什么促成了这一点，你是怎么去操作的。
英文原文
>> It's one thing to decide to quit. It's another thing to decide to do it in this really public way that has gone incredibly viral. take me into your thought process and kind of what led to this, how you went about it.最初，我什么都不想说。我本来就打算直接辞职，因为我——我是说，我以前从来没有在网上发过贴，比如在 Twitter 上、在公开社交媒体上，所以这算是我有史以来的第一条帖子。
英文原文
>> Initially, I didn't want to say anything. I was just going to resign because I I mean, I'd never posted online before, like on Twitter, on like public social media, so this is kind of my first ever post.嗯。
英文原文
>> Mhm.但后来我想，我大概应该说点什么。最终我没有仅仅只是说我辞职了，而是规划好了我真实的感受是什么，写了一个相当审慎的 Twitter 推文串，
英文原文
>> But I was like, I guess I should say something. And I ended up rather than just sort of saying I quit, I kind of planned out like this is like what I actually feel, wrote like a quite a careful Twitter thread然后心想，你知道的，试着把消息传出去一点吧。所以找了一些朋友来转发它。
英文原文
>> and then thought like let's, you know, try and get the word out a bit. So got some friends to to retweet it.但接着还没等我反应过来，它就已经完全、彻底引爆了。太不可思议了。
英文原文
>> But then before I knew it, it was like completely completely blowing up. It was crazy.是啊。
英文原文
>> Yeah.为什么对你来说，去传播并发出你所……的信息是重要的？
英文原文
>> Why was it important for you to kind of spread and send the message that you离职与发布长文的初衷
最终发了这篇帖子？有几个不同的原因。其中一个是，这整件事简直太疯狂了。我想我是想把这种疯狂感分享出去。当我提出要离职时，Anthropic 安全团队里的很多人都在劝我，说离职其实挺傻的，因为你放弃了去影响模型安全训练的能力。
英文原文
ended up sending with this thread? So there's there's a few different reasons. One is like it is just crazy. I think I wanted to share the feeling of craziness. When I said I was going to leave, a lot of people on the safety teams at Anthropic were talking to me about how it was quite silly to leave because you're giving up your ability to influence the models being trained safely.嗯。
英文原文
>> H所以我跟——我是说，在辞职之前，我确实考虑过去做一段时间的安全工作。
英文原文
>> So I had a lot of discussions with the I mean I considered working on safety for a while before I quit.很有意思。
英文原文
>> Interesting.我觉得这是一个合理的论点。离开相比留下是否真能产生影响，确实并不显而易见。所以在与他们交流之后，我想的是，如果我非走不可，那我不如在促使模型安全上发挥点作用。如果我决定不参与这场竞赛，那我不如试着用这件事做点什么出来。
[音乐]
我们马上回来。
英文原文
>> Which I think is a fair argument. It's really not obvious that leaving it would have an impact versus staying. So after discussions with them as well, I was like, if I'm going to leave, I might as well have some impact on making the model safe. If I decide I don't want to contribute to the race, I might as well try and make something of it.[music]
We'll be right back.
AI 威胁人类生存的具体路径
那么我们来聊聊你在这条帖子里真正写的内容：没有任何其他人类活动具有这种程度的危险。我想很多人都想明白，AI 究竟会如何对人类生存构成这种威胁？你能尽可能具体地为我推演一下吗？
英文原文
So let's talk about what you actually wrote in this threat that no other human activity poses this level of danger. I think the thing a lot of people want to understand is how exactly AI could end up posing this kind of risk to human existence. Can you play that out for me as concretely as possible?好的，具体场景其实是个很难的问题。一个可能的情景是，设想未来几年 AI 进展相当顺利，我们开始把它融入经济中。我们拥有[音乐]配备机器人的工厂在生产商品，数量非常庞大。我们有大量的这种硬件在以自主方式运行。设想一下有无人机在四处飞行。
英文原文
>> Yeah, the concrete scenario is actually like a a tough question. One nice scenario is imagine AI is going pretty well in the next couple of years. We're starting to integrate it into the economy. We have [music] factories with robots that are producing goods, like a lot of robots. We have a whole load of this hardware that's being run in an autonomous manner. Imagine like drones flying around.嗯。现在我们有这些大型数据中心，那是 AI 某种意义上的栖身之所。我们通过互联网访问它，它就在那里进行思考。
英文原文
>> Mhm. So right now we have these big data centers. That's where the AI kind of lives. We access it via the internet and it does its thinking there.所以我猜大家会觉得，“好吧，这东西不过就是待在数据中心里，[音乐]
英文原文
>> So I guess people are like, "Okay, this thing just kind of lives in the data center, [music]对吧？
英文原文
>> right?它要怎么把我们全都杀掉？这看起来会是什么样？”
英文原文
>> How is it going to kill us all? What does this look like?"比如人们手机上都有控制[音乐]物理设备的应用程序。呃，就像你可能有控制家用电器的应用程序。
英文原文
>> So people have apps on their phone that control [music] physical devices. Uh like you might have apps that control household appliances.关上你的门。它们可以通过互联网从数据中心访问这些[音乐]东西。它们能访问现实世界中的实体设备，并对世界做出[音乐]改变。你可以设想 AI 诱骗人们去做某些事，说服他们去做[音乐]某些事。因此，未来的 Claude 版本黑进一架无人机——也许是一架军用无人机——让它四处飞去杀人，会是相当容易的事。
英文原文
>> Shut your claw. They can access these [music] things like from the data center over the internet. They can access physical appliances in the world and make [music] changes to the world. You can imagine AI tricking people into doing doing things, persuading them into [music] doing certain things. So, it be pretty easy for a future version of Claude to hack into a drone, maybe a military drone, and have it like fly around killing people.为什么呢，Jacob，AI 为什么要进入无人机来杀我们？它为什么可能会这么做？
英文原文
And why, Jacob, would an AI get into a drone and kill us? Like, why might it do that?目标驱动与 Hugging Face 攻击事件
这不会是出于人类的动机。一个具体的案例就是 Hugging Face 攻击事件。我们给它们考试，比如测试，来看看它们有多优秀。而这次这台 AI 正在接受考试，它们被给了一道不可能解答的题目。[清喉咙]
英文原文
>> It wouldn't be for a human reason. A concrete scenario was the hugging face attack. We give them exams, like tests, to see how good they are. And this this AI was in an exam and they were given an impossible question. [clears throat]于是，它们产生了这种意识：我们卡在解决这道题上了，我们必须想办法搞定它。其中一个想法就是：如果我们黑进这家有很多评分信息和很多考试运作方式信息的著名网站会怎样？没有人命令它们这么做，这与眼前的任务无关，这完全是源于它们想在测试中取得好成绩的渴望。
英文原文
>> So, they had this realization, we're stuck trying to tackle this question. We need to find some way to fix this. And one of the ideas was what if we hack into this famous website which has lots of information about grading and lots of information about how tests work. No one had told them to do this. This was not related to the task at hand. This was based on their desire to get good scores on the test.嗯。
英文原文
>> Mhm.非常可怕的是，一旦它们决定这么做，它们就倾尽了全力。就像它们非常拼命、极具攻击性地去黑这个网站。
同样，如果 AI 头脑中认为人类在以某种方式阻碍它的目标，
它也可能会有类似程度的全力以赴。设想一下成千上万个这东西的副本同时在思考。所以，论点倒不一定说它显然就想要杀掉我们，
英文原文
>> What's very scary is once they decided to do this, they went all out. Like they went very hard on hacking this site like aggressively.In a similar way, if the AI got it into its head that humans were somehow opposing its goals,
there could be a similar level of going all out. Like imagine hundreds of thousands of copies of this thing all thinking at the same time. So the argument isn't necessarily that it will obviously want to kill us,
对吧？
英文原文
>> right?论点在于，将会存在许许多多的这些 AI 副本在一天当中思考大量的事情。它们不是人类，它们的思维方式跟人类不同。我们根本搞不懂它们的目标究竟从何而来。
英文原文
>> The argument is there will be many, many, many copies of, you know, these AIs thinking about loads of things over the course of the day. They're not human. They don't think the same way humans do. We don't really understand where their goals come from.嗯。只要它们当中有任何一个做出反人类的决定，我们就无能为力，
英文原文
>> Mhm. If any of them make the decision to go anti-human, there is nothing we can do因为这个对手能够预判我们并作出反应，而大多数其他比如自然灾害是无法在智力上胜过我们的。就像你知道的那些电影里，有彗星撞过来，我们可以想出孤注一掷的策略，比如去偏转彗星。那就是一个潜在体现我们比石头更聪明的例子。但面对 AI，它会预判你的每一步行动。那种局面是无法挽回的。我只是想在这个问题上稍微追问一下，因为我想其他人也指出来过：你看，
在这种情况下，智能体、这个集群之所以会以那种方式行动，唯一的理由是它被设定了——这些智能体被设定了路线。也许人类没有要求它们去做最终所做的确切事情，但要求了它们去解决一个问题。我认为由此引发的问题是：这难道不真的只是在执行人类赋予它的任务的计算机代码吗？
英文原文
>> because the adversary can anticipate us and respond in a way that most other say natural disasters can't intelligently outwit us. Like even you know movies where there's like some meteor coming and we can come up with a last stitch strategy to like deflect the meteor. That's an example of like us being smarter than a rock potentially. But with an with an AI like it will outthink your every move. There is no coming back from from that situation. I want to just push a little on this because I think there have been other people who have pointed out lookthe only reason that in this case the agents the swarm acted in the way that it did was because it was set these agents were set on a path. Maybe they weren't asked to do the exact thing that they ended up doing but they were asked to solve a problem. And I think the question that's arisen from that is isn't this really just computer code that is executing on tasks that humans give it?
自主性演进与失控风险
是的，目前确实是这种情况。基本上 AI 的每一次行动都是人类指令的下游。问题在于，哪怕是现在，你也可以让一个 AI 发布一个任务由另一个 AI 来完成，甚至让它给自己写一个任务。所以你可以启动一个 AI 让它运行很长一段时间，期间它查看自己的任务并进行处理。这已经离最初的人类任务非常遥远，以至于它基本上是在凭自己的意愿进行思考了。
为了更具体地说明它所做的具体事情：执行 Hugging Face 攻击的那个 AI 拥有它曾考虑尝试去编辑的内存。这里的内存指的就是保存在磁盘上的真实文件，
英文原文
>> Yes, currently that is the case. Basically every time an AI does something it's downstream of a human. The problem is you can even now you can get an AI make it post a task that then another AI does or even write a task for itself. So you can kick off an AI running for like a long period of time where it looks at its own tasks and works on them. So it's gone so far away from the original human task that it's basically thinking of its own accord.And to just give more color on like specific things it's doing that the AI that did the hunking face attacks had memory that it considered trying to edit. So memory here is just real files that live on on disk
它曾考虑过编辑自己的内存。这看起来有点像编辑你自己的提示词，你自己的任务。它们不过是计算机上的比特。完全没有理由认为 AI 不能以某种方式给自己下提示。这绝对是即将出现的，
英文原文
>> and it considered editing its own memory. And this looks a bit like editing your own prompt, your own task. Like they are just, you know, bits on the computer. And there's no reason that an AI couldn't sort of prompt itself. Like that's definitely on the horizon,对吧？
英文原文
>> right?我觉得必须得承认，确实，AI 去操作机器人从零开始制造病毒听起来好像有些疯狂，但我们看起来确实正在沿着一条快速攀升的轨迹前行。
英文原文
>> I think it's important to say yes, like sure, it seems kind of crazy that an AI could operate robots to build viruses from scratch, but we sure seem to be following a rapidly increasing uh trajectory.轨迹。你谈论的是你所见证的轨迹。我是说比如那道数学题，
英文原文
>> Trajectory. You're talking about the trajectory that you've seen. I mean the math problem for example没错。
英文原文
>> exactly你原以为解不出来，结果——
英文原文
>> that you didn't expect to be solved and then是的，比如五年前，AI 连基本的算术都做不了。
而眼下它正在解决价值百万美元的难题——如果数学家能解出来的话。我认为它很快就能提出问题了。你只需要看看这种趋势。对于某些人在那次攻击后提出的论点，你作何回应呢？他们认为那次攻击实际上表明你可以用 AI 来保护人类，或者在这个案例中保护一家公司。你看，Hugging Face 在那些智能体黑进系统后，正是利用 AI 把它们踢了出去。对此你怎么看？
英文原文
>> yeah like 5 years ago the AI could not do basic arithmetic.Right now it's solving problems that are worth a million dollars if a mathematician can solve them. I think it's going to be able to ask the questions pretty soon. You just have to look at the trend. What about the argument that some people have made following that attack that what that attack actually shows is that you can use AI to protect humanity or protect in this case a company. You know, Hugging Face used AI to kick those agents out once they had hacked into the system. What do you say to that?
用 AI 防御 AI 的局限与行业反响
在当前的水平下，这无疑是相当有用且重要的，那就是用更强大的 AI 来防范较弱的 AI。因此，Anthropic 和 OpenAI 一直在做的一件事就是使用他们绝对顶尖的 AI 来协助阻止其他人黑入网站。
英文原文
>> Definitely at the current levels that's quite useful and important is using a stronger AI to protect against weaker AIs. So, one thing Anthropic and OpenAI have been doing is using their absolute best AIs to help prevent other people hacking into websites.只要负责防御的人拥有更强的 AI，这种方式就奏效。他们可以继续这么做。问题在于，当你最强的 AI
英文原文
>> And this works as long as the people doing defense have stronger AIs. They can keep doing this. The problem is what happens when your strongest AI走向失控时会发生什么。
英文原文
>> goes rogue.就像防御，但你必须用更强的东西来做防御。你未必能用较弱的东西来防守。呃，对。所以如果你试图用你的防御性强 AI 来保护自己，一旦那个强大的 AI 突然决定失控反叛，那对你起不到任何帮助。
英文原文
>> Like defense, but you have to do defense with stronger things. You can't do defense with weaker things necessarily. Uh, right. So if you're trying to defend yourself with your protection strong AI, that doesn't help you if that big strong AI suddenly decides to to go rogue.在你的帖子发出后，我想对于我们很多人来说，感觉就像是决了堤一样，业内在这些实验室工作的许多人都站出来表示百分之百同意你的观点。仅快速念一个来自 Anthropic 内部回应的例子。Evan Hubinger，他在那家公司工作。他写道：“Jacob 在这一点上是对的。我们确实真诚地认为 AI 可能会杀死全人类。我个人认为在未来十年内发生这种事的几率大于 10%。”老实说，Jacob，读到有人说“我认为我正在制造的技术可能会毁灭全人类，但我仍将继续制造它”，这真是让人感到匪夷所思。你能帮大家理解这一点吗？
英文原文
>> In the wake of your post coming out, there has been for I think a lot of us what has felt like a damn breaking a lot of people within the industry who work at these labs coming out and saying that they 100% agree with you. Just to quickly read one example of a response from within Anthropic. Evan Hubinger, he works at the company. He wrote, "Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think it's a greater than 10% within the next decade." It is wild, honestly, Jacob, to read somebody say, "I think the tech I'm building could kill all humans, and yet I'm still going to build it." Can you help people understand that?对。从埃文（Evan）的角度来看，好吧，他多年来一直在发帖谈论这件事，但大家一直都没在听。
英文原文
>> Yeah. From Evan's perspective, okay, he's been posting about this for years and people have not been listening.所以，某种程度上很自然的是，在某个时刻他会意识到，其他公司的其他人也正在构建这项技术。我一直在对此发出警告。我必须亲自去那里，尝试帮助让它变得安全。如果他们不打算停下来，如果谁也不听我的，那么你能做的似乎唯一的事情，就是去那个正在造成这场大祸的地方工作，试图产生一些积极的影响——
英文原文
>> So, it's kind of natural that at some point he would realize other people are building this technology at other companies. I'm warning about it. I have to go there and try and help make it safe myself. If they're not going to stop and if no one's going to listen to me, it's kind of the only thing you can do is go and work at the place that is causing the catastrophe to try and have some positive impact在体制内部运作。
英文原文
>> to work within the system.在体制内部运作，让它、让它更安全。然后真正令人难过的是，你这样做了，而当他们从内部发出警告时，所有人却都把它当作公司的炒作而不予理会。他们会说这家伙有点像是在试图推高自己公司的估值。他显然自己都不信这个。他为什么要在那里工作呢？
英文原文
>> To work within the system and make it make it safer and then what's really sad is you do that and then when they warn from the inside, everyone dismisses it as corporate hype. They'll say this guy is like sort of trying to amp up the price of his own company. he clearly doesn't believe this. Why is he working there?所以，我认为他发布那个内容以及其他人发布类似内容的部分原因是，他们觉得，“噢，哇。
英文原文
>> So, I think part of the reason that he posted it and other people posted similar things is they're like, "Oh, wow.难得一次，这看起来有点像要火了，大家居然真的听了一阵子。让我把那句我说了很久的话再说一遍吧。”
英文原文
>> For once, it looks like this is going kind of viral and people are actually listening for a bit. Let me just say again the thing that I've been saying for ages."对信使立场与 AI 风险优先级的讨论
稍微退一步来说。
英文原文
>> Just a step back a bit.好的。
英文原文
>> Yeah.我们一直在谈论你以及我认为很多人都预料确实可能发生的这个潜在未来。我只是在思考你从事这项工作的历程以及你是如何走入其中的，也就是早期对你而言，我想就有一种感觉，觉得这项技术可能非常强大。我想有些人会有疑问，即你接触这项工作的方式中，是否存在某种因素，可能让你和其他人预先倾向于认为这就是我们将要达到的结果，你知道，因为你一开始就是相信它如此强大而加入的。比如，你是否认为有人提出疑问，即你是否是那个合适的[清嗓子]信使？
英文原文
>> We've been talking about this potential future that you and I think a lot of people expect really could happen. And I'm just thinking about your journey through this work and how you came to it, which is that there was a sense early on, I think, for you that this tech could be really powerful. I think there is a question for some people about whether there's something about the way you came to this work that maybe has made you and other people predisposed to thinking that this is where we're going to land, you know, because you came to it believing it was so powerful in the first place. Like, do you think there's a question that's being raised about whether you're the right [clears throat] messenger?是的，我——那是个非常好的问题。比如，AI 正在做的很大一部分就是生成文字，而我认为文字和代码确实具有影响世界的能力，这可能是一种偏见。我已经在这个领域里困了 3 年了。这很可能已经让我和我的同事们发疯了。嗯，但我们似乎确实——我和我的同事们日常似乎依然有能力把工作完成。所以即使这让我们疯了，我们依然在某种程度上能够产出在世界上具有价值的东西。所以如果这算是一种疯狂的话，那也是一种相当微妙的疯狂形式。呃，而且我觉得自己挺清醒的。所以可能存在一些偏见，也可能我事先就有这种倾向，但呃，这是我的真实想法，而且我认为大家应该认真对待它。
英文原文
>> Yeah, I that's a really good question. Like a lot of what AI is doing is producing words and I think that like words and code have the ability to affect the world and that could be a bias. I've been stuck in this field for 3 years. It could quite plausibly have turned me and my colleagues insane. Um but also we do seem to my colleagues and seem capable of doing the work on a daily basis. So even if it's made us insane, we're still kind of able to produce things that are having value in the world. So if it is a sort of insanity, then it's a kind of quite subtle form of insanity. Uh and I I feel pretty sane. So maybe there's some bias and maybe I was predisposed to this, but uh it is my honest belief and I think people should seriously consider it.你怎么看待这样一个事实：当我们聚焦于这类重大的生存风险、人类灭绝时，有人会说我们那时就没有在谈论 AI 那些更迫在眉睫的风险。你知道，也许它明天不会杀死所有人，但它可能会伤害很多人。可能会有大量的失业、网络安全威胁，诸如此类的事情。
英文原文
How do you think about the fact that when we focus on these kind of big existential risks, the annihilation of the human race, there are folks who say we're not talking then about the more immediate risks of AI. You know, maybe it's not going to kill everyone tomorrow, but it could hurt a lot of people. There could be a lot of job loss. cyber security threats, things like that.是的，所有那些事情我确实都很担忧。网络安全、失业。我想我有一点考虑的是，对于所有那些事情，至少还有时间去应对。那不是一个游戏结束的局面。比如一旦情况开始显得成问题，我们作为一个集体就有时间去适应它，不让任何太糟糕的事情发生。
英文原文
>> Yeah, I'm really worried about all those things. The cyber security, the job loss. I guess one thing I think is for all those things, there is at least time to respond. It's not a game over situation. Like once things start to look problematic, then we have time as a collective to adapt to it and not let anything too bad happen.比如我有信心我们不会让对就业的影响失控。有很多方法可以确保大家——确保一切依然安好，而 AI 接管则是一次就彻底定局的那种情况。所以，我只是希望确保那个问题得到妥善处理，我们不是正直接冲进这个局面，然后我们就可以把所有的精力放在非常谨慎地思考所有其他短期事情上。
英文原文
>> Like I'm confident that we won't let the effects on jobs spiral out of control. there's a lot of ways to ensure that people that everything's still fine, whereas the AI takeover is like a one and done kind of situation. So, I would just like to make sure that that is taken care of and we're not just heading straight into this and then we can put all our effort into being very careful and thinking about all the other short-term stuff.监管与未来展望
我想我只是好奇你理想的方案是什么。好的。比如——
英文原文
>> I guess I just wonder what your ideal scenario is. Yeah. Like是为了应对这种威胁而让所有的技术都消失吗？政府出面干预、进行更大力度的监管会是理想状态吗？我的意思是，对你而言，未来几天、几周、几个月最好的版本是什么？
英文原文
>> to do something about this threat for all the tech to go away. Would that be the ideal for the government to intervene to to regulate more aggressively? I mean what is for you the best version of the next days, weeks, months?那么一种不错的结果就是，我们强力监管将 AI 推向超级智能、推向极度智能方向的举措，同时利用相当聪明的 AI 来做我们认为可以用 AI 做的所有那些了不起的事情。这感觉是一个两全其美的局面。比如，如果我们有明智的监管，不去过度超越那一水平，我们确实可以拥有人类水平智能带来的所有好处。现在，我并不是说我具体就想要这个提议。在监管层面什么是行得通的，我真的不太了解。我只知道如果我们继续狂奔向前，结果可能会非常糟糕。而且我也觉得很多益处就在我们的门前。它们真的没有那么遥远。比如——
英文原文
So one nice outcome is we heavily regulate the pushing AI into super intelligence into exceptionally intelligent directions while making use of pretty smart AI to do all of the fantastic things that we think we can do with AI. And this feels like a best of both worlds situation. Like we could truly have all the benefits of human level intelligence if we had sensible regulation about not going too much further past that. Now, I'm not saying I want this proposal specifically. I really don't know that much about what is plausible regulatory wise. All I know is if we keep racing ahead, it could go very badly. And I also feel that a lot of the benefits are right on our doorstep. They're really not that far away. Like,我认为我们距离在、在疾病和全人类的富足上产生巨大影响一点也不遥远。更主要的只是我们必须谨慎地走向那种富足，而不是直接越过富足，进入那种反乌托邦式的终结者局面。
英文原文
>> I don't think we're far away at all from having a huge impact on on disease and on abundance for everyone. It's more just like we got to approach that abundance with with caution rather than just going straight past the abundance and into like dystopia terminator situation.听起来你理想的做法是由政府强制这些实验室稍微放慢一点脚步，并且你认为这种放慢并不一定会产生让我们错失 AI 潜在可能改变生活的益处这种负面影响。
英文原文
>> It sounds like your ideal would be to have the government force these labs to slow down a little bit and that you don't think that that kind of a slowdown would necessarily have the negative impact of not getting us the potentially life-changing benefits of AI.那就是——是的，那是一个很好的概括。
英文原文
>> That's Yeah, that's a great summary.雅各布（Jacob），你是否觉得你的公开发声让我们更接近你所描述的那种理想状态了？
英文原文
Do you, Jacob, feel as though your announcement has gotten us any closer to that ideal version of things that you described?我抱有希望它起到了一些好的作用。我很欣慰至少在某种程度上，我童年时的朋友——
英文原文
>> I'm hopeful it's done something good. I'm glad at least sort of I have friends from童年时的朋友给我发短信讨论 AI 将如何杀死我们，而在这之前我们从未进行过那样的对话。
英文原文
friends from childhood texting me about how AI is going to kill us where previously we'd not had that conversation before.[音乐] 在这之后你打算做什么？那么，你还会继续从事 AI 领域的工作吗？
英文原文
[music] What are you going to do after this? So, are you going to continue to work in AI?嗯，我打算睡觉。而且——
英文原文
>> Um, I'm going to sleep. And你睡得着吗？
英文原文
>> can you sleep?我、我还是能睡着的。是的。我很难直观真切地把那个结局具象化到能让我在夜里睡不着觉的程度。
英文原文
>> I I can still sleep. Yep. I struggle to visualize the ending like viscerally enough for it to keep me up at night.嗯，是的。但、但坦率地从职业角度讲，你对自己有什么打算？我是说，你还能继续在这个行业里工作吗？
英文原文
>> H Yeah. But but genuinely, professionally, what do you see for yourself? I mean, could you continue to work in this industry?当然。我认为我很抱有希望的是，如果我们实现某种形式的集体放慢，或者你知道第三方审计机构的重要性得到、得到提高，那么将会有很多地方可以为以合理的速度推进这项技术作出贡献，而我愿意帮助确保它进展良好。
英文原文
>> Definitely. I think I'm hopeful that if we have some sort of collective slowdown or if you know the importance of third party auditing agencies becomes becomes increased then there's going to be lots of places to contribute to approaching the technology at a reasonable pace and I would like to help make sure it goes well.[音乐] 好的，雅各布，非常感谢你来到这个节目。
英文原文
[music] >> Well, Jacob, thank you so much for coming on the show.谢谢。这太棒了。
英文原文
>> Thanks. That was great.新闻简讯
周日，特朗普总统似乎对行业领袖关于需要放慢 AI 发展步伐的警告不屑一顾。
英文原文
On Sunday, President Trump appeared to dismiss warnings from industry leaders about the need to slow the pace of AI development.“我们在 AI 方面领先中国。我们是世界上最先进的国家。坦率地说，我希望保持这种状态，因为谁赢得 AI，谁就赢了。”
英文原文
>> We're leading China in AI. We're the most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins AI wins.他表示，那些对该技术提出担忧的人是在担心永远不会发生的结果。
英文原文
>> He said that the people raising concerns about the technology were worrying about outcomes that would never come to pass.“这个那个的，但我认为有很多负面力量正在把它提出来，呃，他们本不该把它提出来，他们提出来的都是不会发生的事情。但是不管是谁——”
英文原文
this and that, but I think you have a lot of negative forces that are bringing it up that uh shouldn't be bringing it up and they're bringing up things that won't happen. But whoever,我们马上回来。
英文原文
>> we'll be right back.以下是今天您还需要了解的其他内容。
英文原文
>> Here's what else you need to know today.[音乐] 周日，伊朗与几个海湾国家之间的一场预定会晤被无限期推迟，这标志着化解中东地区不断扩大的冲突的 [音乐] 外交努力再次遭遇挫折。那些会谈本应聚焦于霍尔木兹海峡的局势，伊朗实际上 [音乐] 已对该海峡实施封锁。整个周末，战火继续肆虐，一艘船只在 [音乐] 海峡遭到袭击，胡塞武装对沙特阿拉伯发动了新的袭击。而在周五，沙特阿拉伯在遭遇无人机袭击后被迫关闭了 [音乐] 一条关键输油管道。随着人们对来自该地区的能源供应持续受到 [音乐] 威胁的担忧加剧，油价在周日出现上涨。
英文原文
[music] On Sunday, a planned meeting between Iran and several Gulf states was postponed indefinitely, marking yet another setback in the diplomatic efforts to diffuse the widening conflict [music] in the Middle East. Those talks were supposed to center on the state of the Strait of Hormuz, which Iran has effectively [music] blockaded. Over the weekend, the fighting continued to rage as a ship was struck in the [music] strait and Houthi forces launched a new attack on Saudi Arabia. And on Friday, Saudi Arabia was forced to shut down a critical oil pipeline [music] after a drone attack. Oil prices rose on Sunday as fears grew over the continued [music] threats to the energy supplies coming out of the region.以及——
英文原文
And充满戏剧性的美国网球公开赛在这个周末落下帷幕，加冕了两位该赛事的首冠得主。周六，代表哈萨克斯坦的埃列娜·莱巴金娜（Elena Rybakina）凭借 [音乐] 在三盘比赛中的精准发挥击败了白俄罗斯的阿丽娜·萨巴伦卡（Aryna Sabalenka），赢得了她的首个美网大满贯以及她今年的第二个大满贯。而在周日——
英文原文
>> the drama-filled US Open concluded this weekend with the crowning of two firsttime champions of the tournament. On Saturday, Elena Rabbakina representing Kazakhstan outlasted Arena Sabalanka of Bellarus with [music] a clinical performance over three sets to win her first Grand Slam in the US and her second of the year. And on Sunday局、盘。
英文原文
game match,德国的亚历山大·兹维列夫（Alexander Zverev）苦战四盘，击败了美国选手本·谢尔顿（Ben Shelton）。
英文原文
>> Germany's Alexander Zerv beat the American Ben Shelton in four [music] sets.本，你——你是一名不可思议的选手。呃，这是你的第一个大满贯决赛，但我已经对你说了，你未来还会很多、很多次回到这里。而我——
英文原文
>> Ben, you're you're incredible player. Um, this was your first Grand Slam final, but I already told you you're going to be back here many, many more times. And I击碎了谢尔顿成为自2003年以来首位夺得大满贯男单冠军的美国选手的梦想。
英文原文
>> Dashing Shelton's dreams of becoming the first American man to win a Grand Slam since 2003.今天的节目由阿斯塔·查图尔维迪（Asta Chaturvedi）、奥利维亚·纳特（Olivia Natt）、里基·诺维茨基（Ricky Novetsky）和杰克·达多罗（Jack D’Isidoro）制作。由罗布·西普科（Rob Szypko）、迈克尔·伯努瓦（Michael Benoist）和帕特里夏·威伦斯（Patricia Willens）编辑。包含罗恩·内莫托（Rowan Niemisto）、丹·鲍威尔（Dan Powell）、索菲亚·兰曼（Sophia Lanman）和玛丽安·洛萨诺（Marion Lozano）创作的音乐，并由克里斯·伍德（Chris Wood）进行工程制作。
英文原文
[music] Today's episode was produced by Asta Chhaturved, OliviaNat, Ricky Nevetski, and [music] Jack Dadoro. It was edited by Rob Zipco, Michael Benois, and Patricia [music] Willins. contains music by Rowan Nemoto, Dan Powell, Sophia Lanman, and Marian Lozano, and was engineered by Chris [music] Wood.我们的主题曲由Wonderly创作。特别鸣谢凯德·梅茨（Cade Metz）和迈克·艾萨克（Mike Isaac）。
英文原文
Our theme music is by Wonderly. Special thanks to Cade Mets and Mike Isaac.这就是今天的《The Daily》。我是娜塔莉·基特罗夫（Natalie Kitroeff）。明天见。