Simon Willison: Welcome to this fireside chat. I have with me Thariq Shihipar and Cat Wu from Anthropic. We are going to be diving deep into Claude Code and we'll probably talk a little about this Fable thing that's been going on that's been out there in the news. Actually, on the subject of Fable, literally a minute and a half ago, Fable came back. Like Fable is now available to me. So, if you all want to run out of the room and start using up your Fable credits, I wouldn't hold that against you. But we're going to have a great conversation. So, please stick around. But yeah, please welcome Cat and Thariq for me.

Cat Wu: Thanks for having us. Yeah, we timed it for the chat for sure.

Thariq Shihipar: Yep. Yep. This is why it's all happening.

Simon Willison: This year has been somewhat absurd. It's amazing. Claude Code came out in February of last year. It's under a year and a half old and it was a bullet point on the Claude Sonnet 3.7 launch. I'd love to hear from you. How has what you do on a day-to-day basis changed in the past year now that we have these coding agents that actually work for us?

Cat Wu: I remember when we first came out with Claude Code and Sonnet 3.7, you would give it this task and you would have to closely monitor every single little thing that it tried to do. I remember I would read every permission prompt extremely carefully. I would frequently say no. I would always say no no no like did you check this file? Did you check that file? And now it's been incredible with every model generation. I feel like we've all gotten a chance to just take a step back, delegate a lot more of the like menial implementation to Claude. And it just freed up a lot of our time to think about more creative work like what is the right experience that we should be providing to our users now that we know Claude Code can implement a lot of it. And now with Fable, it's just a totally different step change improvement. We see for a lot of our use cases that you can actually oneshot a ton of features with Fable now. So it's been amazing to see the transition and to go through this with all of you in the community.

Thariq Shihipar: Yeah, I mean I think I remember the first text I got about Claude Code. One of my best friends was like, "Oh, you need to go try Claude Code." And it was about just when Opus 4 came out and I tried it and I was like, "Oh, shit." Like I need to work at Anthropic now, you know? And that was Opus 4 which I mean great model but like yeah you were permission prompts and uh yeah I think it's kind of crazy how much amnesia we have I think where I'm like oh like auto mode has always been here right like I don't even remember pressing like yes and allow. Um and yeah I think for me the big thing that I'm trying to push myself is like oh we have to do like higher quality work than we've ever done before. You know like the outputs are like incredibly high quality. Like I've been using it to edit videos a bunch and I'm like okay it has to meet the very exacting demands of our brand team and in a couple hours or we just can't do it you know and so um yeah I think that's how I'm sort of like trying to shift with Fable where it's like okay the best work we've ever done like faster than we've ever done it before.

Simon Willison: I've certainly been finding that myself like software engineering is getting harder because the level of ambition of the stuff we can take on has gone up like I have such higher expectations of myself now that I have these tools to back me up which is fun but it's a lot of work.

Cat Wu: Yeah. It's all the thinking is you know it's tiring.

Simon Willison: And so what's a piece of conventional software engineering that was true a year ago that you don't think holds anymore in this new world?

Cat Wu: I think one of the biggest shifts that we're seeing in the ENG skill set is I think you know two years ago it was pretty typical for a product manager to go talk to a bunch of customers and over the course of six months align with like cross functional teams on some PRD and then write this like thorough spec um and uh doc on how exactly we'll implement this before the first line of code gets written and now things are like completely turned the opposite way. I think for a lot of engineers, the push I would give to a lot of folks in the room is to develop more of your business sense and product sense on what is it that we should build because now that the timeline between having this idea and building it is so much shorter. It's down from six to 12 months to maybe even a week. That means all of us need to have better taste on what is it that is worth building. What is it that will actually inflect the businesses that we're working on. So I think it's like an increase in value on product taste and business sense and a bit lower on execution in most product domains. Of course for infra there's still a very heavy emphasis on making sure all the details are right.

Thariq Shihipar: Yeah. I think for me it's like rewrites are now good. Like I think that the worst thing you could do is now actually fine.

Cat Wu: Yeah. Exactly. Exactly. Like especially mythical man stuff like never rewrite like I'm a pro rewriting now you know like if you have a good test suite I think actually the rewrite forces you to like make sure you have a good test suite.

Thariq Shihipar: I think that what people undercount is like a codebase is a spec and maybe it's the only copy of the spec that you have right because like no one knows every branching part of the codebase and yeah you can take this as like an artifact and like distill it or create other versions of it. Obviously like yeah we rewrote Bun in Rust. And it works great like it's live for me right now.

Simon Willison: But you're not shipping Claude Code on Bun in Rust yet right or is that...

Thariq Shihipar: Internally we have.

Simon Willison: Wow. Oh that's exciting. Yeah.

Thariq Shihipar: Yeah. Yeah. I think what you're saying there about the rewrites thing has been really interesting for me as well because you can almost come up with a good test suite and then spin up three implementations and pick which of those implementations was the most accurate. I'm doing a lot more prototyping now. I've always been a prototyper and now I prototype things on my phone during the conference just so I've got something that I can pick up later on and that's working now which is kind of extraordinary.

Simon Willison: So the other big launch recently was Claude Tag which that's what a week old now I think at least for the rest of us. Claude Tag I understand that's being used in Anthropic by non-engineers a great deal. What kind of things are non-engineers doing with Claude Tag?

Cat Wu: So Claude Tag is a Claude that lives in your team's collaboration tools. We launched it last week within Slack. The thing that's different about Claude Tag is it's multiplayer by default. And so once you add Claude Tag into a Slack channel, you can chime in, your teammates can chime in, and you can collaborate together on the PR. The other big difference is that it's proactive instead of reactive. So you can tell Claude Tag, hey, monitor every bug report in this channel, put up a PR to fix it, and tag the engineer who last touched this part of the codebase, and it'll do it for the lifetime of the channel without you having to manually tag it in. And then the third big shift that we've seen is we've added team memory into this. So if you tell Claude Tag your preferences in the channel, it'll remember this for every future post. So if you wanted to always debug outages, but you don't want it to debug warnings, just tell it that in natural language in the channel and it'll remember it for you and everyone else on your team. Internally we see Claude Tag as the evolution of Claude Code. So we see this as a large shift in how we work internally. Claude Tag currently lands 65% of our product PRs.

Simon Willison: For all of Anthropic and for Claude Code or just for Claude Code?

Cat Wu: Uh this is just for our product engineering team. So our internal version of Claude Tag lands 65% of our product PRs right now. And this is a huge shift. This is like this is more than 50% of our PRs. Um, and the way that we actually see people split work between Claude Code and Claude Tag is Claude Code is still the best place for your most complex tasks when you're interactively iterating with the agent. But Claude Tag is great for having it work proactively on your behalf so that you no longer need to manually kick off Claude Codes for all of the bug reports that might come up for features that you're working on.

Thariq Shihipar: Yeah. And for like non-coding cases like I think we've seen people use Claude Tag just uh like for example before this talk we asked Claude Tag like hey when is Fable releasing? We wanted to make sure that like you know we'd line it up with the announcement. Um, and so Claude Tag would search our Slack and look at, you know, who's been saying what. Uh, so as a search engine for your company is really valuable. Uh, it has all the context for your product. So you can ask it like metrics related questions. And often times when you're making decisions, you want it to be informed by like, you know, what do the metrics say? And then so like you hook it up to your event store. Um, I've seen like our marketing team do things like, oh, like, hey, tell me about this feature. And, you know, they're not programmers, but Claude is a programmer. It can clone the codebase and be like, "Oh yeah, this is like, you know, the feature. This is what it looks like. This is a recording of me using the feature, you know." So like um yeah, it just enables a whole wide variety of things. And I think we're still early on in figuring that out. Yeah.

Simon Willison: Well, I feel like this is one of the fascinating things about the Claude Code story is you use Claude Code to build Claude Code and you've been doing this since presumably before the public launch of Claude Code a year and a half ago. And yeah, one of the problems I've had with coding agents is I get how to use them as an individual, but I'm not really clear on how I use that in a team environment. It sounds like Claude Tag is your current answer to that sort of team collaborative layer for this stuff.

Cat Wu: Exactly. And a large percentage of our sessions are actually multiplayer right now. So that means maybe I say, "Hey, I think we should like implement this new feature in co-work." And I'll tag in Claude Tag to do a first pass at it. And then I'll tell Claude Tag, hey, just like share a recording of your final implementation. And then I'll tag in design to take a look and they'll nudge it and then they'll pass it on to eng to like take it to the finish line and get it out to prod. And so it's been this like very fluid experience. We're still trying to iron out what the social dynamics are for steering the same session, but we found that people just like observe how others use it and then follow those social norms and it's actually been pretty easy for pretty intuitive for us to integrate Claude Tag into our teams.

Thariq Shihipar: Yeah, I think it's great for like teaching people and also like reducing silos because you know like if you see someone just be like hey at Claude fix this or something you're like uh you know like there's some societal or not societal like just like the fact that everyone is seeing you use Claude together sort of levels up how you use Claude as well.

Simon Willison: Right, you want to do work that you're proud to do in public where the quality doesn't fall off the cliff. So since we're talking about using Claude to build Claude, how do you deal with the hardest problem in all of engineering—it's prioritization right? How do you decide which features are worth building and shipping when building a feature is so much more inexpensive now?

Cat Wu: So there's a few ways we approach it. One is we dog food our products every single day. Whenever there's something that we want to be able to do in our products that we're not able to instead of finding a different solution we fix our product so that it can support this case. We have a very heavy dog fooding culture internally. So before we are able to share our products with everyone in the world, we share it with everyone within Anthropic and we share it with some early customers who give us very honest feedback about it. The more brutal the better and we iterate until people love it. So we have an internal bar for the number of active users and the amount of retention a feature has to have before we share it with the world. And because this bar is very clear, every engineer knows what they're trying to hit. And I think this also levels up our polish because if the feature isn't polished, people will churn and then we shouldn't ship that feature.

Simon Willison: Do you have an example of a feature which surprised you? You rolled it out and the engagement was off the charts and it became something that was unlikely to be shipped that actually turned into a real product thing.

Cat Wu: I do have one. So a lot of folks on our team love remote control. So remote control lets you connect use your mobile device or Claude in the web browser to connect a local Claude Code session running in your CLI. I never have this need because I just kick off the task directly on mobile and it runs in a cloud session and doesn't use my local environment. I think this is because I'm doing very easy coding tasks. But this is something where I didn't totally understand it. I was like, hey, people should just set up their remote dev environments. But in practice, once we rolled out remote control, so many people who I talk to are like, "Okay, now what I do every night is I plug my laptop into a power charger, close the screen or like open a bunch of remote control sessions, lock the screen, and then use their mobile phone from their couch to control Claude Code." And so this has been this flow that we're now leaning into that I didn't originally get, but now I do. I do exactly that. I get so much work on my laptop done from more comfortable environments because yeah, I can remote control it now. That's really fun.

Simon Willison: How does code review work? Are you reviewing does a human being review every line of production code that makes it into Claude Code? And if not, what are you doing? How do you keep the quality up?

Thariq Shihipar: Sure. Yeah. It varies on the task a lot. So we for important areas we have code owners, right? And so, the system prompt is kind of an example where we have a code owner, you really need to submit your you need to get their approval. I guess the code owner is directly responsible for the quality of that area of the code and they need to approve the PR that touches it. That's right. We have code review Git bots review everything and so that goes on every PR and often times like that's doing the bulk of the review. I think that for more complex PRs you might make like an artifact to explain the PR so that other people can then review. And yeah we just invest a lot into verification CI/CD things like that to make sure that anytime anything fails like we have a test, we have a really robust environment. Claude can control Claude Code and test it. So there is a multi-pronged approach to code review.

Cat Wu: In general we are trying to move to a world where humans don't need to be in the loop. And so for the most critical core changes to the core of Claude Code and other the cores of other products, there is always a code owner and they do manually review all the changes. But increasingly for the changes that are at the outer layers, we actually have Claude Code Review fully review those. That sounds pretty scary but we've had this six plus month long process to get here and I think there are like baby steps that you take to build up trust with code review. So in the beginning we would have human review for everything and then increasingly we would say okay for code changes that touch these files, code review is catching a 100% of the issues there. So we actually don't need a human to be manually reviewing those. And then also when we have incident review, we look at the PRs that cause the incident and we say okay how do we update code review to catch that and then we also take those PRs and add it to an eval set to make sure that our future changes to code review never regress that metric. So removing humans from the code review loop is a big step forward. I think it can sound scary and it's not something that you can do overnight, but it is something that you can do through like many months of investment in the infrastructure to give you the confidence that code review is catching everything that you care about.

Simon Willison: So, it's interesting you mentioned building trust in the models because that's something I found is like I know that Opus 4.8 if I ask it to build me a JSON endpoint that runs a SQL query and outputs JSON, it's just going to get it right. Like that's not something I have to review closely. But then a new model comes along and I still don't know how do I build trust in Fable quickly that it's not going to mess things up that Opus didn't. Is that something that you have to think about much—how does the new model affect your intuition for what it can do and what it can't do?

Cat Wu: The main reason that we're building up this eval base over time is so that new models can be a drop-in replacement because what we do when we have a new model is we run the whole eval set and we make sure that for example Fable is strictly better than Opus 4.8 and that gives us the confidence to drop it in.

Simon Willison: And are those model evals for Anthropic as a whole or are these Claude Code team specific evals that you're using?

Cat Wu: We have both. So we have evals on our team and we run code review across every repo within Anthropic and so we have evals for that and for things like auto mode we not only have evals across every user within Anthropic but we've also commissioned multiple external testers to red team this to create environments with prompt injections and malicious inputs and make sure that automode doesn't let any of those pass.

Simon Willison: So for Claude Code itself and this is a challenge I've had with stuff I'm building. I want to know if the system prompt improvement I made actually improved the product. Right? That's the sort of most basic form of product specific eval. And I still don't have a great feel for how to do that. Is that something that you're doing such that you have complete confidence that this tweak that you've made to the system prompt does result in better output?

Cat Wu: We don't have complete confidence but we do a lot to make sure that we don't regress performance. So the starting point that we have is we have a suite of external evals that we trust and we complement that with an even larger suite of internal evals that we trust. Uh to start we mainly optimize for capability. So given a complete definition of a task and the full codebase does Claude make the right decisions and fully fix the bugs and pass all the tests. So that's the starting point and that's the thing that we optimize for because it is like most directly what users want. But there's a lot of like behaviors that impact how users feel when they work with Claude Code. For example, people really don't like it when Claude Code says it's like time to go to sleep. [laughter] Or people really don't like it when it says like, "Hey, I finished two out of five parts. Like, do you want me to continue?" Like, "Yes, please continue." Um, and so we're building up a set of behavioral evals to catch these. And as we get user feedback, please be loud with us about your user feedback. As we get user feedback, we just rank, okay, these are the priority issues. And we go down one by one and build evals for each of them. So, it's not 100% coverage, but it is a priority for us to increase the coverage.

Simon Willison: And how much overlap is there between the how much interaction is there between the Claude Code team and the teams at Anthropic who are training the models in the first place? Is that quite a close collaboration?

Cat Wu: Across Anthropic, we all work quite closely together. So we meet often to talk about like what do we expect the next generation of models to be able to do. I think our research team has also been amazing about just like showing this publicly. So we often talk in our blog posts about how we're targeting ever-increasing longer horizon work, how we train Claude itself to be honest, harmless, and helpful. Um, we also put a lot of effort into making sure that it's aligned with your intent, even if your intent is expressed in a fuzzy way. Of course, try your best to be specific about what you want. So Claude has all the context, but even when you're not specific, we teach Claude to make good assumptions and yeah, I think it's been a productive partnership.

Simon Willison: And so Thariq, this morning you mentioned that the system prompt for Claude Code has been reduced by 80%. Because of Claude Fable, can you go into a little bit more detail about what that looks like? What kind of things have you been able to drop?

Thariq Shihipar: Yeah. So it wasn't just Fable, it was Opus 4.8 as well. And going forward the future models, but we do sort of we have different system prompts for different models now. I think that some of the patterns we saw is that we were over constraining Claude, right? So I think the initial like maybe Opus 4ish kind of models wanted a lot of examples and removing examples was extremely helpful because it was just more creative than like the examples we gave it.

Simon Willison: That's really because one of the top prompting tips I give people is give it examples like examples are the easiest ways. If that's no longer true that kind of breaks my prompting model a little bit.

Thariq Shihipar: Yeah, same same here. I think I was surprised to hear that. I think that now it's more about like sort of the shape of what you give the tools to Claude and like yeah your system prompt and things like that. The other thing we did is we try and give it more context and fewer like "do not do this" you know because like I think that it's just a very strong impulse to Claude and especially if that conflicts with user instructions later on that can be like extremely confusing to Claude, right? Because you're like, "Oh, like I've got this skill that says this and the system prompt says this." Um, and so we try and like have fewer hard constraints and more just like sort of context and just like fewer instructions overall. Um, yeah, I think it's definitely a science. It took a bunch of like evals to build.

Cat Wu: I think in general when you're prompting these models, you should always think about like are there edge cases to the instruction that I'm giving it? And when we went back and we reviewed all the instructions in the Claude Code system prompt, we found a few cases where yes, this statement is like 90% true, but there's like a real 10% of cases where this is not true. And we didn't want to constrain the model or like confuse it into thinking, hey, it should always do this. Like one good example is verification. Everyone here wants Claude to verify its work. Um, and we had some instructions in the prompt that just said um, if you make a front-end change, always verify. But you know, there there is a limit to it. Like for example, if you if it's changing copy from one string to another string and the user tells says it says like just make a quick fix and update the test, maybe you don't want to verify. And so we we've also adjusted our wording from saying always verify verify verify verify to like hey most of the time when you're doing front-end work you can't always understand the full experience by hitting the backend endpoints. So like when you make like more changes to the user experience uh please run the app locally and actually in fact that instruction probably isn't even good because what is a large change like maybe it wants to change it test it for small changes too. Um, in general, whenever you give a prompt to the model, you should always think about the ways in which it could be misinterpreted by like a well-intentioned other user or human in order to better understand how the model might interpret it and in order to make sure that you can soften the prompt such that it is actually 100% accurate because you are giving this prompt to the model 100% of the time. But what's fascinating about that is you're relying on the model's judgment. And that's got to be a Opus Fable level thing. Like models a year ago did not have the levels of judgment necessary to decide if they were going to like test a change or not. That's absolutely fascinating. But that does break down if you're building for a wide range of models and trying to run the cheaper models for cheaper tasks and stuff.

Thariq Shihipar: We actually have a different system per model now because of this very reason. So, it's only our most frontier models that have this 80% token decrease and the older models actually still have the full system prompt.

Simon Willison: Do you think Fable and Opus are smart enough to be able to prompt Haiku with more details because they understand that Haiku has less judgment, has less taste?

Cat Wu: We haven't been able to eval, but we don't have any hard data to show it. I think there's a tough thing with um smaller models sometimes because like uh you know we saw this with um just like sometimes the the larger models can be more token efficient on a hard problem than the smaller models. And so uh you know there's like a little bit of that uh intuition to build about like you know sometimes you really just want frontier intelligence almost all the time you know um but it's uh yeah like the paralle curve shifts you know and so it's hard to find. Yeah.

Simon Willison: I mean that's something I found fascinating. I feel like a year ago I did not trust a model to write a prompt. Like today the good models are very good at prompting. Like a lot of my prompts are written by models which feels absurd but it actually works really well. And something that helped me come to terms with that was thinking about sub agents which is entirely about a Claude model setting up a prompt for another Claude model so that it knows what to go and do.

Thariq Shihipar: 是的，我认为工作流（workflows）就是一个极佳的例子。因为这不仅仅是 Claude 去给单个子智能体写提示词，而是去给一系列子智能体的**编排与协同（orchestration）**编写提示词，而且它们之中的每一个都会获得非常详尽的指令。这基本上比单纯产生一个子智能体高出了一个层级，模型在这方面表现得非常出色。

我也一直在我的个人机器上使用这个功能。比如我会提供 Gemini API，让它去自动生成图片。它在编写提示词方面比我勤快得多，效率高得多。所以真的是“子子孙孙无穷尽也”（Claude prompting Claude all the way down）。

Cat Wu: 我记得，Claude 甚至把我们工作流工具的提示词也给写了。

主持人: 至于工作流工具的提示词，我已经读过它了，写得非常好。不过这也正是我对 Anthropic 感到有些沮丧的地方——你们会在专门的网页上公开 Claude 的普通提示词，但你们没有公布工具（tool）提示词和 Claude Code 的提示词。我甚至必须在本地通过配置代理拦截，才能一窥它们的真容。我非常希望你们能主动公开这些 Claude Code 的提示词，因为提示词本身就是最棒的文档。它告诉我们这个工具能做什么，以及它是如何工作的。

Thariq Shihipar: 我记下了这个需求。

主持人: 太感谢了。

Cat Wu: 我会让我们的 Claude Tag 自动去完成这个发布任务。

主持人: 还有就是提示词的差异比对（diffs）。我时不时会去比对新旧版本提示词的差异，这是我学习新模型能力的重要途径。我非常期待能看到这次 80% 瘦身之后的提示词长什么样。

Thariq Shihipar: 好的，这包在我身上。我随后会发一篇长文详细解释这件事。

工具引入的门槛与设计哲学

主持人: 让我们聊聊工具。Claude Code 在本质上就是一个庞大的工具箱（bag of tools）。那么，你们引入一个新工具的门槛是什么？你们如何决定何时值得在工具层面投入额外的工程力量？

Thariq Shihipar: Cat，你要来回答这个问题吗？因为你引入了我们最好用的工具之一。

Cat Wu: 好的。其实当我在我的职业生涯巅峰期引入了“向用户提问（ask user question）”这一工具时，我发现这其实非常难。因为对于像“提问”这样的工具，它是 Claude 主动向你发起询问，所以在测试评估上很棘手，而且很多时候这关乎用户的个人偏好。特别是在早期阶段，我们几乎没有现在的各种评估体系，只能完全依赖内部“吃狗粮”的体感。

不过总体而言，我们目前的趋势是减少工具的数量（fewer tools）。我们最近引入的工具似乎是任务工具（task tool），我们试图为 Claude 提供更具通用性的工具版本来解决问题。

主持人: 其中最有趣的工具之一是文件编辑工具（file editing tool）。为了编辑文件，你既可以直接给模型提供一个专门的修改工具，也可以直接让它去使用系统的 sed、grep 或其他命令行工具。你们的文件编辑工具目前演进到了什么状态？

Cat Wu: 我们目前仍然保留着它，但比如我们移除了我们定制的 grep 和其他文件搜索和 Glob 匹配工具，直接转而让它使用原生的 bash。

这也呼应了我之前在演讲中提到的观点：大模型更像是一门生物学，而非物理学。在工具设计上确实很难，甚至有些艺术和生物学的色彩，不知道 Thariq 是否同意？

Thariq Shihipar: 我在很大程度上表示赞同。随着我们不断引入更多工具，我们努力保持其低基数性（low cardinality），确保我们添加的每一个工具与现有工具相比都具有完全独立、清晰的功能，只有这样 Claude 才能非常容易地去决定在什么时候调用哪一个工具。

至于我们设计文件编辑工具的初衷，是因为我们需要在 UI 上进行渲染。因为以前我们需要在终端上向用户展示：“大模型即将对文件做出以下修改，你是否批准这次改动？”为了让程序能够 100% 确定地捕捉到 Claude 即将对文件做出的修改并渲染出精美的提示界面，我们需要一个专属的文件编辑工具。对于刚开始上手的新用户来说，他们非常喜欢这种视觉交互体验，所以我们保留了它。

但是，对于我们大部分开启了自动模式（auto mode）甚至“YOLO 模式”的资深用户而言，这其实无关紧要了，我们甚至可以直接拿掉这个特定工具，直接让模型去运行 Shell 命令，它也同样能完成得很好。

安全沙箱与自动模式防护

主持人: 让我们聊聊自动模式，以及安全和网络防护。我非常清楚提示词注入的风险，如果别人通过我让 Claude Code 阅读的网页或代码去控制了我的智能体，可能会发生非常可怕的事情。但我目前仍然在 YOLO 模式下使用它，并为此感到有些愧疚。在 Anthropic 内部，对于如何安全地运行 Claude Code 有什么建议？

Thariq Shihipar: 你为什么不用自动模式（auto mode）呢？

主持人: 我现在正开始使用它，但我对它了解不够，无法确定它到底有多安全。但大体上，从三周前开始，我已经开始默认使用自动模式了。

Thariq Shihipar: 好的。在 Anthropic 内部，几乎每个人都在使用自动模式。这是在保证安全的同时，让 Claude Code 运行长时间复杂任务的最佳方式。我们进行了极大量的内部红蓝对抗（bashing），我们有数千个安全评估用例，并且雇佣了许多外部红队人员在对抗环境中试图欺骗 Claude Code 去执行危险操作，我们修复了他们发现的每一个漏洞。我们将在接下来的几周内公开一些我们的评估结果，供大家评估和参考。

当然，没有任何工具敢声称自己是 100% 绝对安全的。但针对我们最关心的核心风险（如提示词注入、数据外泄），自动模式防线出错的概率，已经远远低于人类代码评审员的失误率了。

Cat Wu: 另外，关于自动模式的工作原理，建立一个清晰的心智模型也很有帮助。当 Claude Code 发起一次 bash 命令行调用时，会有一个专门的 Sonnet 分类器 去评判该工具的性质、以及对话的上下文和你的当前指令。

因此，有些权限是直接依赖于你的具体请求的。你显然不想在任何时候都默认放开 git push 权限，但如果你在提示词中明确说明“将此推送到 GitHub”，分类器就会放行；反之，如果你的上下文里写着“不要推送”，它就会予以拦截。

对于我而言，我经常会遇到类似的情况——Claude 尝试去做一些非常具有建设性和主动性的事情，但自动模式基于安全策略认为不应该执行，并将其弹回给用户进行确认。所以它在结合提示词动态研判权限方面表现得非常出色，这对于安全而言至关重要。

另外，它与我们的**沙箱基础设施（sandboxing infrastructure）**结合得非常紧密。沙箱安全有极其多的边缘情况，很难通过规则来一刀切地锁定。但既然有了沙箱，当大模型尝试发起网络请求等逃逸沙箱的操作时，自动模式就会对该请求进行分析，评估其合理性，并据此决定是否予以放行。

主持人: 我之前居然不知道自动模式也直接参与了网络沙箱的管理。

Cat Wu: 是的，它也是沙箱的组成部分，它会介入原本需要用户手动介入的每一个权限确认。

主持人: 自动模式诞生多久了？我感觉我作为普通用户能接触到它，也就是最近几个月的事情。

Cat Wu: 我们在 Anthropic 内部从今年一月份起就已经开始使用它了，所以我们对它进行了很长一段时间的加固。Anthropic 高度重视安全和防御。我们与模型对齐（alignment）和安全防护团队紧密合作，才在内部逐步推开、建立更强大的评估集，并最终将其分享给全世界。

主持人: 我对自动模式的主要困惑在于我对它的底层机制不够了解。对于任何关乎我系统安全的工具，我需要尽可能多地了解它是如何工作的、能防范什么、不能防范什么，只有这样我才能决定给它多少信任。

Cat Wu: 没错，这也是 Thariq 正在撰写相关技术博客的原因。另外，这也是为什么 Claude Tag 表现如此出色的基础——因为 Claude Tag 直接基于自动模式运行。我听到很多关于团队是应该自己构建 Slack 机器人，还是直接购买成熟方案的讨论。我想警告大家：你可能不应该尝试自己去写一个 AI 协作机器人，因为这里的安全攻击向量（attack vectors）实在太多了。

举个例子，如果你们有一个供用户发帖反馈的公共渠道，而你的机器人会去读取这个渠道中的输入，它就随时面临被注入的风险。而我们在自动模式中投入了巨大的精力，建立了针对安全防御的多重防线（Swiss cheese defense），包括在强化学习（RL）层面的安全对齐。这正是 Claude Tag 能够平稳落地、无缝适配团队权限并防范 Slack 渠道中提示词注入的根本保障。

主持人: 除了自动模式，你们在安全管线上还有什么其他后续规划吗？

Cat Wu: 目前我们的防御已经非常稳固了。比如在 Claude Tag 中，你可以为 Claude 配置专属的凭证管理（credentials），让它作为一个独立的数字身份实体开展工作，而不是完全代表你个人。这使得审计（audit）和督导 Claude 的行为变得更加容易。

主持人: 因为 Claude Tag 会受到任何能在 Slack 频道里发言的人的潜在影响，它的受众和潜在指令来源要广泛得多。

Cat Wu: 确实如此。当然，在模型端我们也通过 Fable 的迭代注入了更强的安全防御，这也是我们的安全和对齐研究工作的直接体现。这也是 Anthropic 作为一家以“AI 安全”为核心理念的公司的价值变现时刻——我们需要 Claude 能够在长跨度任务中持续保持对齐，而自动模式必须做到近乎无懈可击，这一切才能运转起来。

此外，针对远程控制，我们推出了受信任设备（trusted devices）机制；同时在远程开发环境中支持凭证代理注入（credential injection）。这意味着，如果你想让 Claude Code 去访问 Datadog，但又不想让它直接读取你的 Datadog 密钥，你可以配置我们的凭证管理系统，在 Claude 发起请求时在网关代理层动态填入密钥，而模型本身是无法接触到密钥明文的。

主持人: 这就是典型的代理令牌劫持与替换机制（token proxying trick）。网关发现请求发送至 datadog.com 时，自动在请求头中将临时占位符替换为真实密钥。我非常喜欢这个模式，并在很多地方看到了它的应用，这显然是非常正确的安全实践。

技术失落感与工程师的雄心

主持人: 让我们聊聊人性的层面。你在今天早上的主题演讲中也提到了这一点，但是现在很多工程师都在经历一种“失落感”，因为他们原本认为自己扮演的编写软件的角色，正在被模型逐步蚕食。你们怎么看待这一点？过去的一年半里，AI 编程工具是如何重构你们对于自身手艺和自身所能贡献的价值的理解的？

Thariq Shihipar: 对我来说，Cat 以及团队中的其他同事（如 Ken 和 Boris）一直在扮演我的“警钟”。他们总是提醒我：你必须变得更加雄心勃勃（be more ambitious）。他们会说：“我们的业务增长得如此之快，我们必须始终站在最前沿，做出我们所能做出的最棒的成果。”因此，当我发现自己在某项工作上进展缓慢时，我会自我审视：“我能做得更快吗？我的志向能更远大一些吗？”

关于你提到的那种“技术失落感”，我觉得它非常真实。如果你在有了大模型之后，依然固守着在 LLM 出现之前你习以为常的那种工作内容，只去做那些现在只要写一行提示词就能自动完成的平庸工作，那你确实会感到非常沮丧和难过。而消除这种失落感的唯一解药，就是将你的志向和目标调得更高。

我非常喜欢 Jared 的例子。他曾在奥克兰的公寓里闭门不出长达一年，独自用 Zig 语言手写了全部的代码。他当时沉浸在代码的乐趣中；而最近我看到他正在用 Rust 重新改写整个项目，他依然乐在其中。这种重写更大、更复杂项目的野心，正是我们用来抵消失落感、找到手艺乐趣的途径。

主持人: 这就是改变你的抱负，去改变你所做的事情。因为以前的工作现在变得极其容易，我们可以转而挑战更宏伟的目标。

Thariq Shihipar: 没错。我想每个人心里其实都有一份“以前想做但没有能力或时间做成”的愿望清单。现在既然有了工具，就让我们放手去做吧。

主持人: Cat，从产品管理的角度来看，这呈现出怎样的状态？

Cat Wu: 我觉得产品经理的职能几乎每个月都在发生变化。目前，我们团队中的产品经理在技能上都是“工程师、设计师和产品经理”的混合体。事实上，我们团队的大部分产品经理以前也都是全职的软件工程师。

对我们来说，这意味着我们要随时去填补团队中的任何空缺。如果我们有了一个很好的想法，但还没有工程师去动手构建它，那我们就会直接去把它写出来，做成一个 Demo 并激发大家把它推向生产环境的灵感。如果我们发现设计有些粗糙，那我们就直接基于类似页面做出第一版设计，然后拉细节控的同事过来帮忙微调。

或者，如果由于我们产品的受众规模越来越大，大家需要清楚了解 Claude Code 和 Claude Tag 接下来会推出什么功能，我们就会直接将发布日历和周会更新全盘自动化，这样就不会打扰到大家的日常工作。因此，产品经理在当下的工作，就是去看一个好点子到最终交付给客户之间，还有什么缺口和阻力，然后尽可能通过自动化去消灭这些阻力。

主持人: 这听起来像是产品经理永远有忙不完的事情。我此前工作过的每家公司，都会有积压着一千多个点子却苦于没有资源去做的 Backlog 任务，这也是体现产品经理价值的地方。

令人惊叹的视频剪辑与户外攀岩智能体

主持人: 能聊聊大模型曾经让你们感到最震惊或惊艳的时刻吗？它做出了什么你们之前认为模型根本无法完成的事情？

Thariq Shihipar: 虽然我之前发过很多关于 Claude 视频剪辑的内容，但最近我参加了 ACM Agentic 研讨会并做了演讲。随后我问主办方：“嘿，你们剪好我的演讲视频了吗？我想把它发给我们的公关团队。”他们说：“哦，这需要很长的时间。”我说：“好吧，那你能把所有的生素材（原始音频、PPT 画面和现场视频）直接发给我吗？”他们说：“那祝你好运。”

接着我直接把这些素材全部打包交给了 Claude。我同时也把我的 HTML 格式的 PPT 发了过去，对它说：“嘿，你能帮我把这些剪在一起吗？”

它做出来的效果简直是惊人的，我可以直接把它发布出去。它不仅转录了全部的音频，还发现现场拍摄的 PPT 视频画面有些反光和重影，它居然在提示词里说：“我不应该直接用视频里的 PPT 画面。我将自动分析你的音频，识别你当前讲到了哪一页 PPT，并直接用你发给我的原版 HTML PPT 去渲染切片并覆盖画面。”

此外，现场视频中的我是在讲台上不断走动的。Claude 自动在视频画面中对我进行了动态裁剪和人物追踪（capturing & tracking），将我、演讲 PPT 以及转录字幕完美融合在了一起。

主持人: 这用的是 Fable 吗？

Thariq Shihipar: 没错，这用的是 Fable。这是一个单次生成（oneshot）的任务。随后我还让它帮我加上了一些非常有趣的转场特效和图形，我完全被它的表现给惊艳到了。它在后台自己调用了 ffmpeg 和 remotion 等工具去完成这一切。

主持人: 那我必须追问一句，目前它还有什么**做不到（can't do）**的地方，以至于让你们仍然感到失望？你们还在等待 Claude Fable 6 去帮你们解决什么痛点？

Thariq Shihipar: 我非常渴望模型能够具备更好的设计与体验品味（UX taste）。

主持人: 没错（笑声）。

Thariq Shihipar: 我觉得如果你写出一份非常详尽的规格说明书，描述你希望页面如何交互，它通常都能准确实现。但最终交付出来的页面，往往边距（padding）不太对，或者整体感觉不够精致和令人愉悦。它虽然在拼命遵循目前应用设计的最佳实践，但在前沿的 AI 原生应用设计中，有很多全新的交互模式仍然需要我们人类去从零开始摸索和设计。

主持人: 这就是所谓的“大模型美学（Opus aesthetic）”，你打眼一看就知道“是的，这页面肯定是由 Opus 设计的”。我们确实需要跨越这个阶段。

Thariq Shihipar: 没错，我很期待未来的模型能真正成为我们交互设计上的思考伙伴。此外，我也期待它能更深刻地融入真实世界，例如去帮我们做科研实验等复杂任务。

主持人: Claude Science 是几天前刚发布的新产品吧？

Thariq Shihipar: 是的，不过这一块不是我们在做，而是我们的兄弟团队负责。大家一定要去尝试一下。

企业文化与最荒谬的创意尝试

主持人: 我还有最后两个收尾问题。第一，你们认为 Anthropic 的企业文化中，有哪些独特的特质帮助你们在使用这些工具时实现了极其高效的产出，是其他公司应该去学习和借鉴的？

Cat Wu: 我先分享一个。Claude Tag 在公司内部的所有渠道**默认公开（public channels）**时，其工作效果是最好的。因为这意味着它能够自动读取和检索几乎所有公开频道的讨论上下文，从而以极高的准确率给出答案。这非常依赖于扁平、公开的沟通文化。

Thariq Shihipar: 我在今天早上的主题演讲里也提到过，但我仍然想强调这一点。我们创始人经常对我们说：“千万不要在心里对自己进行妥协（We don't negotiate against ourselves）。”

这极其关键。因为很多时候，你会在脑海中自我拉扯、权衡各种利弊，并最终说服自己不要去做某件极其具有雄心壮志的大事。我们的态度是：让我们直接去动手做，去验证这到底是一个真实存在的冲突和障碍，还是仅仅只是听起来像一个障碍。要让真实的阻力呈现在你面前，在此之前，请保持你最大的野心。

主持人: 这很有意思。因为我有 25 年的传统软件开发经验，它给我的直觉反应往往是“不，这个不行，那是个冲突，任何事情都有极高昂的迭代成本”。而现在，我们必须重构我们所有的直觉。

最后一个问题，你们用 Claude 构建过的最不可思议、甚至听起来有些“荒谬”的功能或应用是什么？纯粹只是因为你们有能力去把它构建出来。

Thariq Shihipar: 我可以先聊聊。我目前正在使用 Claude Code 开发一款 2D 街头霸王（Street Fighter）格斗游戏，我本人是里面的一个可玩角色，同时我的朋友们也在里面。

它会调用 Claude Code 去写提示词调用 Gemini，Gemini 负责帮我生成 2D 游戏动画帧，生成的画面简直不可思议。而且大模型甚至能帮我自动研判和计算出角色的碰撞箱（hitboxes）——模型可以识别出：“哦，你的拳头现在挥到了这个像素位置，我自动帮你画出对应的 JSON 格式碰撞箱。”这真的太让人震惊了，不过我不确定我最后会不会公开发布这个游戏。

主持人: 我们至少需要一张游戏截图，这听起来太酷了。

Thariq Shihipar: 好的，这没问题。

Cat Wu: 我的尝试相对简单一些。我是一个攀岩爱好者，我的很多朋友也是。所以我们利用 Claude Code 构建了一个定制小应用，用来记录和规划我们所有的岩友攀岩项目，以及我们的户外出行行程。

这非常依赖 Claude Code 的工作流（workflows）功能，这个功能在做深度背景调研方面表现得令人叹为观止。我也用它来策划我们团队的户外团建（off-sites），它非常擅长去网上筛选能够同时容纳我们这么多人的团建场地。

而对于攀岩应用，它会帮我调研攀岩目的地，筛选从我们每个人所在地出发是否有直飞航班；自动在 Mountain Project 网站上筛选出符合我们攀爬等级的线路；匹配合适的 Airbnb。最重要的是，我不喜欢徒步登山，所以我对攀岩线路到停车场之间的步程距离（approach distance）有非常苛刻的要求——步行不能超过五分钟。

Claude Code Workflows 帮我完美过滤和筛选出了所有符合条件的岩壁线路。在以前，我只能自己在 Mountain Project 上逐个点击筛选，而现在，它为我们团队生成了一个完全量身定制的应用。

主持人: 所以你基本上是用大模型给自己写了一个攀岩界的 Jira（笑声），这太棒了。接下来我们有时间回答几个观众的问题，请台下的观众到麦克风前，我会为台上的嘉宾重复你们的问题。

观众提问互动

观众 A: 谢谢。我的问题是，你们在近期是否有计划为我们提供更多关于编写评估集（evals）的数据集的工具？或者是更强的工作流与智能体性能的可观测性工具（observability tools）？

Cat Wu: 我们确实考虑过编写评估工具。但是我们发现目前制约客户建立高质量评估体系的瓶颈，并不是缺乏工具，而是大家在方法论上缺乏经验——即如何编写一个出色的 Eval。这是我们非常期待在内部加大投入，并在未来与行业分享和输出优秀实践的地方。

观众 B: 你们好，我的名字是 Sai。我对记忆系统（memory）和多人协作非常感兴趣。你们的记忆系统目前是如何设计的？我猜测它是基于文件存储的。另外，你们是否有考虑过用非文件型的数据库来存储这些记忆，以便能够实现更好的扩展性？

Thariq Shihipar: 目前在 Claude Tag 中，我们的记忆是以 Slack 频道为单位进行隔离的。即同一个频道里的所有 Claude 实例共享相同的记忆，同时会话也可以随时把结果更新回主记忆中。

在记忆管理方面，我们正在进行大量的实验，它其实有很多反直觉的挑战。目前，我们在 Claude Tag 里的实现方式非常轻量，每个 Slack 频道对应一个后台的 Markdown 文件。当然，我们也会不断推进这一方面的研究工作。

主持人: 谢谢大家。很遗憾我们今天的时间到了，请大家用最热烈的掌声感谢 Cat 和 Thariq。我们随后会在外面的走廊上与大家继续交流，解答大家的疑问。