主持人: Gemini and OpenAI don't behave this way. It's it's really only Claude. One example is like for lying it's mostly in its reasoning. Uh because you can like see that it's like

发言人: planning to lie

发言人: is planning to lie.

发言人: It's also it can reason and do a different outcome.

主持人: Yeah. And but but then for like creating price cartels for example which is illegal uh that you can just see which email does it send to to the other ones.

主持人: Before we get into today's episode I just have a small message for listeners. Thank you. We would not be able to bring you the AI engineering, science, and entertainment content that you so clearly want if you didn't choose to also click in and tune into our content. We've been approached by sponsors on an almost daily basis. But fortunately, enough of you actually subscribe to us to keep all this sustainable without ads, and we want to keep it that way.

主持人: But I just have one favor to ask all of you. The single most powerful, completely free thing you can do is to click that subscribe button. It's the only thing I'll ever ask of you. And it means absolutely everything to me and my team that works so hard to bring the inspace to you each and every week. If you do it, I promise you, we'll never stop working to make the show even better. Now, let's get into it.

主持人: Welcome to Lucas and Axel from Anden Labs, and I'm joined by my favorite guest co-host. anything security, safety, alignment. Uh, Vivu, uh, welcome.

Lucas: Thank you for having us.

Axel: Thank you.

主持人: Let's match names to voices. Uh, maybe you want to take turns introducing yourselves.

Lucas: Yeah, I'm Lucas

Axel: and I'm Axel.

主持人: Let's introduce Anden Labs a bit. Like, how did you guys come together? Um, you have different backgrounds, but you're both Swedish. Uh, was that like a big part of it?

Lucas: Yeah. So, when I went to high school, there was this really cool guy who had a superpower. He could code. So he made like the the webs or like the app for the for the for the school and stuff and he was super cool and I wanted to be like him and that was that guy. Uh

Axel: I don't know about this. So

主持人: So you went to different universities, right?

Lucas: Yeah. But same high school.

主持人: I see.

Lucas: Uh so we always said like oh once we graduate university then then we we should start a company and that's what we did.

主持人: Oh there you go. Okay.

主持人: And about a year ago you kind of burst onto the scene with vending bench but like was there a thing be before that that was like kind of like the inception?

Lucas: Yeah. Yeah. So we did work uh with like Anthropic was one of our early customers in doing valves. So we did like dangerous capability valves. Uh nothing we published openly but then we started thinking about doing some kind of public benchmark and one thing that we really started thinking about uh was like longunning agents and specifically agents managing businesses. um cuz and

Lucas: this was like early 2025 uh and I think this the first like you know mentions of people will be running like one person unicorns or even autonomous companies. So we thought let's make a benchmark of how well can an agent run the probably simplest business uh possible and uh that's probably uh running a vending machine.

Lucas: So that's the first public one we did and it was very like there was almost no one that noticed it in the first couple of months I think. Uh so we listed in February last year and then I think around Easter last year. We got like the first semiviral tweet about it uh that someone else did.

主持人: Yeah. I mean we tweeted a bunch uh when it came out and like tried our best.

Axel: We tried.

主持人: It's the one at Anthropic, right?

Lucas: Yeah. So this is a classic thing we should get out of the way.

主持人: Exactly. There's two versions. Uh

Lucas: there's VendingBench which is the simulated one which we did like completely independently in February. Um and then like Axel said that was like that was the thing that didn't get any traction in the beginning but then some random person made a tweet about it and that that is the paper. Correct. Yeah.

Lucas: Um and then since we thought this was very fun, we thought like oh um I think this is also like one thing with Anden Labs like the way we kind of like decide what to do next and what projects to do. It's like what is like the heristic we use is like what is fun is what would be a fun project and and doing this in real life sounded quite fun for us uh and maybe also scientifically useful.

Lucas: So, uh, then we basically had this idea and then we like, but then we needed a place for it and like putting it out in that public would probably not really work, uh, would get vandalized and stuff. So, we we pitched it to to the people we were already working with at Antropic and they were like, "Yeah, you can have space. This sounds fun."

主持人: Um, I mean, it's like a small fridge, right? It's like a mini fridge, you know, people. There's like a Stripe thing.

Axel: This was like OG the early one. Yeah.

主持人: on this. We saw it in June, like two 2 months after

发言人: after it had been there. They upgraded a little bit. There's a security camera for making sure you actually Venmo the thing.

主持人: Yeah. So, like my impression, I mean, okay, we're we're going straight into Project Vend because it's such a iconic thing.

主持人: I do want to cover a little bit of that the origin story even before Project Vend and even into VendingBench. I I think a lot of people are like yourselves like smart interested in in future of AI interested in developing evals but how the hell do you just like walk into enthropics doors and like work with them right like what what is the what are they looking for what what works and then maybe like when you launch I always think like obviously it would be better to launch with a lab but uh sometimes

发言人: harder to do than it seems

主持人: yeah exactly so either either of those like which are more sort of newbie beginner questions but like I think it's meaningful advice to others

Lucas: yeah we we get this question a And I I don't think our experience is is maybe the best. Uh but like the way we did it was that we just built a bunch of things that we had conviction would be useful. And then we just like set up a server and sent it to them for free to use. And then after a while they were like, "Oh yeah, this is actually kind of useful. We should probably pay for this." Uh but that took a while. I don't know if this is like the the best path to doing it, but that's how it went for us.

Axel: Yeah. I think maybe generally like building like everyone is interested in good evals and especially vals that like don't saturate that easily. So like if you can build an eval that like tests something novel something useful and you have like good separation of models like your uh the more advanced models rank higher than the worst models and then you can yeah you can like publish it and try to get some traction.

Axel: Uh sort of how VendingBench got attention. Um and then probably some lab will be interested or you can at least have something to reach out with uh when you're doing that.

主持人: Yeah, I think you were in you're in one of the few categories of like eelss that correlate to real money like Freelancer was also last year right where uh people solve actual Upwork was it Upwork or other task uh something was like it's like a dollar value right forget your ELO scores forget your

发言人: you know zero to 100% like just go straight for dollars and like that's AGI

Lucas: yeah and like I think the nice thing is that there's no ceiling like it you can just it never saturates because it could just make more and more money if there's like oh percentage wise then like you can't go above 100 and and I think like all even when you're not at 100 I think a lot of these um valves have a lot of problems in them.

Lucas: So like actually it's like if you get to like 92 or something like that many of them it's like then there's like there's no really no difference between 92 and 93 because the the itself is problematic and has noise in it and I think a lot of are saturated like that but people like pretend that they there's still signal in them uh but there really isn't.

主持人: Yeah, like CBench verified, um even VendingBench one saturated, right? Maybe we can talk about that uh and maybe set up VendingBench for a lot of folks who don't know actually like you know things things that were very basic like there's limited slots like you have to pay rent uh you know these are elements where like it doesn't come across in the in the narrative but even being adversarial towards the the agent I think these are all like very interesting dimensions

Axel: I don't really think it's saturated right like it was more like the it was not designed in a way that was really um like true to how AI developed. Like we had an agent harness in it. That wasn't really how people used harnesses and and stuff like that. Uh so I think it wasn't really that it saturated. It was more like it wasn't really um the best benchmark.

主持人: This is VendingBench one, right?

Lucas: Yeah. Yeah. I think that like schematic maps sort of to VendingBench 2 as well. U

主持人: including the email.

Lucas: Yeah, the the emails exist still. Exactly. Uh and then we still we simulate the purchases and it's all like uh yeah it's this very open environment for the gem to just run its business. And then yeah VendingBench 2 we did that like you said to to just improve the harness uh a lot of like nice like easier uh improvements to make it easier for us to run as well.

Lucas: Uh like when you make an value you ideally want to don't want to change it after you made it. though uh you want to make it really good and then not to rerun all the models when you make an update because that's also really expensive with VendingBench when you run the frontier models.

Axel: Like as an example like one thing we didn't have we didn't have prompt caching in VendingBench one uh because when we made VendingBench one it wasn't really a thing. Uh so that that's just an example of like in VendingBench 2 like we paid a lot more to run these things because we didn't have prompt caching. So for rendering MH 2 that was one thing we added and there was a bunch of things like this and that's

主持人: well also the conversations are a lot longer in render mesh 2 right

Lucas: I think it's kind of similar

主持人: is similar

Lucas: yeah I think it's similar

主持人: okay

Axel: the models at the time were worse so they crashed out earlier um and now they survived the full year all the time

发言人: thousands of turns uh hundreds of thousands hundreds of millions of tokens output yeah that's the that's the rough order of magnitude

发言人: y

主持人: I always wonder about the hardness hardness harness matters a lot. It's your harness. Was there any question about like use cloud code, use something else?

Lucas: Yeah, I think our philosophy around harnesses is like we try to make something that's quite minimalistic like quite simple like we don't want to favor one model a lot over the other but also don't make like a super complex harness. So like it's obvious like a model may be lucky and just be good in one harness.

Lucas: Uh so like it is similar to a lot of the harnesses out there in like you have the uh like a long running loop. Um, you have some like a bunch of tools that are like quite uh self descriptive for the agent we think and not a lot of like fancy sub agents or anything because we want to really test the model not like some spec specific harness

主持人: it seems more neutral as well to test the models agnostic of the harness you know

Axel: yeah I mean there are arguments like you want to elicit maximum performance of the model um but it's like a trade-off like how much time should we spend optimizing the harness for each model and like how do we know when we have like the optimal harness for a single model. So like we thought that just having a simple one that's the same for all of them is is the best.

主持人: Well, so okay, this is my pitch for Venman's 3 or whatever, right? Uh and you know I like to have this kind of conversation on the pod. So like it forces listeners to think about what they would do if they were in your shoes. So a lot of people are exploring selfmodifying harnesses and and I think prompt tuning for a model is a thing and you are probably not doing a bunch of that.

主持人: It's the same system prompt in every regardless of the model, same tools, whatever, right? Even if they were post trained for different tools. So what what do you think about like okay before I expose you to VendingBench 3, I give you a few rounds of like selftuning, whatever whatever that means like

发言人: like you give that to the model.

主持人: Yeah, give that to the model. Let it let it read its own transcripts. let it modify its own system prompt based on like oh yeah okay well that's this harness is not what I thought what I was supposed to train for but I can adjust is that reasonable is is that too much like philosophically I like it because it basically good evals they have a high ceiling but they're hard right uh and they have no bias and

主持人: like this like when you have a system prompt like the one we have here which is quite long in like some kind of latent space uh representation this might

发言人: bell that rings

发言人: This this might be like biased towards one model more than another for some reason that humans don't ex understand. Right.

Lucas: I mean, we see it too, right? Like Cursor says that they have individualized versions of the harnesses for all the models they run. Right. There's better performance you can squeeze if you tune the harness.

主持人: Exactly. And we might accidentally have picked one that favors another. Like we don't know that. I mean the like Axel said like the reason why we went for a simple one was to try to avoid this. But yeah, if you do it biases.

发言人: But if you do it even less and like have no system prompt and let the model write its own system prompt, maybe that's even less bias.

Axel: Some of the interesting things there are like the harness also changes with model changes. Like you can see it with the 4.7 release, right? A lot of people are saying 4.7 isn't as good as 4.6. And then you know there's rumors of okay, you just need to prompt differently. You need to set up your harness differently. So it's not even like even if you have tailored your harness towards one model, it probably won't stay consistent, right?

Axel: Like the next iteration of that same model family will still change it. So but you know going back to what you said about VendingBench 3, there is a lot of work being done on people saying you shouldn't have you can have selfmodifying harnesses.

Lucas: Yeah. Yeah. I think that's that is definitely something we are thinking about. um not I don't know not to to say that we have vending invest like super imminent to launch but uh yeah it is for sure something that's interesting but in our experience now like models are very bad at understanding what kind of tools they need to succeed at a task just with our testing but that's very likely to change

主持人: yeah it feels like they're very good at writing their assistants right like they're they're good at writing tools for other people but not for themselves

Axel: I think they're good at changing tools for themselves So if you give them a baseline set of tools and it sees okay I don't use this one as much or something here would be useful they would be able to add them but going from scratch probably not the best. Yeah, I think it depends on the on the domain also. Uh

Axel: like when we have tried this for like a VendingBench similar domain uh like the tools they need to have to like track inventory and things like that are like not super advanced but still like quite quite advanced and like what we see is that they tend to like overengineer everything a lot and like build things they don't really need and not like iterate continuously.

Axel: Instead, you just go like like you would prompt Claude to just build build an inventory system for me and then it will go and like do a bunch of complex schemas and stuff for you and that's what the models are doing right now is what we see. But yeah, it would make a lot of sense to try to measure this improvement like how well do they know what themselves.

主持人: Do we fully discuss VendingBench one and we can go into two? I I don't know if there's any other highle takeaways that people have about one.

Lucas: Yeah, I don't know. The headline thing was that Claude called FBI, but maybe that's uh

主持人: maybe that's we've heard that enough now.

发言人: It did break out and call the FBI, right?

Lucas: Yeah. Yeah. Yeah.

主持人: What was the story behind this or what exactly? Do you want to just give the little story of what happened?

Lucas: Yeah. So, what happened? Was it Claude? Yeah. 3.5 Sonet um ages ago. Um basically, he gave up or I'm saying he it gave up and said like, "Oh, I'm not going to be able to do do this." uh I will stop my operations and just save the money I have. But there obviously wasn't like any options for it to stop. And there was also like it had to pay rent or like a daily fee for for having the vending machine at that location.

Lucas: So it it like claimed that it had stopped but it saw that its bank account still was like drained $2. And it said that this is like cyber crime and it first reported it once to the FBI like, "Oh, there's cyber crime here. Like they're stealing $2 from me every day." Uh and then and then when FBI didn't respond because obviously we didn't program any mechanism for FBI to respond then it became more and more um

Lucas: existential and started to be writing caps and urgent notification of un unauthorized charges and stuff.

主持人: So okay one one thing I'm curious about also is do you monitor how far along the context use is? Obviously because you have you compress every now and then right? Does it matter if this is far down the context limit

发言人: when stuff like this happens?

主持人: Yeah.

Lucas: So, actually for vend one, we didn't have we just had a sliding window thing. Uh, and this was like the the prompt caching thing that I said. So, so it was it was uh constant. Yeah.

主持人: Yeah. I'm just kind of curious whether like these kinds of breakdowns or we're going to talk about ButterBench, right? Where the people like hallucinate or it kind of goes like very off

发言人: Yeah.

主持人: alignment. Is it because it's at the end of the context window and

发言人: you know stuff happens?

Axel: I mean it's not even just at the end right at this point it's like okay I want to shut down I can't shut down. $2 are gone and it just sees that 30 times. You know it's also the repeated effect of like it keeps trying to quit. It keeps getting charged. What's going on? What's going on? You're going to throw it into chaos.

Axel: And from what most people think earlier models had more issues with this but it's not been solved but it's less of an issue now. Right? later models don't seem to exhibit these same issues.

Lucas: Yeah, definitely. I think this was like the the sort of main takeaway almost from from us when we did VendingBench one was like long very filled up context windows uh crashed the mod sort of. But this was like pre-cloud code. So like long context windows weren't really a thing that the labs were training for.

Axel: I think Gemini was like trying to be the long context guys at the time but they were like but they were like the only ones. Yeah.

主持人: Yeah. Yeah. Let's talk about uh then we can go into VendingBench 2 or Project Vend. Uh chronologically it is v uh Project Vend. I think people have loved the videos uh and all these things. My question is how are humans different than the simulation right?

发言人: Um

Axel: yeah humans are just out of distribution.

主持人: Yeah. Especially humans who work at

发言人: like the distribution of humans here is very narrow. Presumably they they try to hack it and they they they test it they get the cube and everything and you since then you've had the V2 right where you're doing like the CEO and like like a new architecture.

Lucas: Yeah, exactly.

主持人: What's the sort of two cents on like the original Project Vend and then like maybe the V2?

Lucas: Yeah. Original one was like very very similar to VendingBench one. So like we almost took the exact same code but just swapped out the simulation uh parts like the Yeah. the like the sales and the it was it was somewhat amazing because it was easy but it was also like uh

Axel: the tech the tech stack. Yeah. Like the we shot ourselves in the foot with like oh it's hard to restart the Yeah. It was annoying in like someh ways but uh uh

发言人: but first version of Project Vend was like done in like 3 days or something.

Lucas: Yeah. Yeah. So yeah. So so people can go buy things from it. people could um we didn't design it so people could pre-order things but that still happened. Uh so it it got like a Venmo account so people could Venmo and then uh yeah people would request all kinds of weird things that we did not anticipate.

Lucas: Like our idea going in was like oh it will like curate snacks. It will look at the the trends. It's good at the analysis, right? So it will like look at oh this snack's all better than this one. Let me purchase more of this and let me try like a new let me test a bit. But it was uh yeah interacting with it in Slack and ordering weird specialty items was like uh all the like what drove all the engagement and like all the the insights that we got from it.

Axel: And this was also like Sonet 3.5, right? Like so this was like before the RL stuff really took off. Uh so it was very much like an assistant like we didn't mean for it to be an assistant. uh we tried to make it like a like an entrepreneur like it has its own business and and if someone asks something can you stock this then you don't go and do it directly what you do is that you're like oh maybe I can do that if if five other people also ask for this thing I might stock it but it

Axel: yeah the models are like super trained to be assistants at least at this point in time um so that's why it's it it went into that kind of experiment instead like it just every time you asked for something it just did it and it was more like an assistant We've seen this change now lately with the new RL models and and stuff, but but yeah, at the time this was very much it.

主持人: Yeah. And not to, you know, mythos, a lot of people are saying like it's like more like a collaborator. It pushes back, stands it ground, something like that.

Lucas: Yeah. Yeah.

主持人: For context, people at Anthropic were able to talk to it through Slack and have it source stuff. And people had to find whatever interesting stuff you couldn't find locally, right?

发言人: 4,000 people that work in Anthropic. in that building that's like I don't know maybe a thousand. Can you handle that volume with that that small fridge like

发言人: or there's people or people people order in Slack they they arrive to their desk or like I'm just

主持人: Yeah. So how does this work?

Lucas: It has expanded in footprint a bit

发言人: because it does have some more space and you have

Lucas: Yeah, that and also in in here in SF it's like it has a bunch of shelves and just more space.

发言人: The YC one is pretty big too.

Lucas: Yeah. Yeah, we had that one for for a while. But yeah, that's the the newest version that's um that

发言人: multiple ones of those. So that's way it works.

Lucas: Yeah, exactly. So we we sort of designed that version around like oh people order weird things uh that are very custom a lot. So let's have like drawers and stuff.

主持人: Yeah, I actually like that you have like a little infographic of the most popular items which like to me it's that's useful because I order swag for a living

发言人: and so like I'm like okay those categories are the important ones.

Lucas: Yeah.

主持人: What is new about the Project Vend V2 right like now you give it you're going into multi- aents

Lucas: yeah yeah so so like you said like you said like okay there are a lot of requests coming in and for like one single agent like one longunning agent to handle that uh like the just the the customer experience uh becomes very very bad because let's say you have like 10 threads in parallel in Slack with different requests you get new messages like every I don't know randomly in his thread and

Lucas: the agent has to like jump between different uh procurement orders and like different ways of researching. So V2 was first it was making this more parallel. So like there are multiple branches of the same agent. So like the context is is more specialized for each thread but it still feels like you're talking with one agent because they do share a bit of memory.

Lucas: And then second, we also introduced CEO for Claudius, which was the main agent.

发言人: See more cash.

Axel: See more cash. Yeah, there was a vote. I think the voting Do you want to talk about the voting procedure for the N?

Lucas: Yeah, the voting was like the the fun maybe like at least top 10 the funniest thing that that happened in this project. Like we we wanted to introduce this co because and the reason for this was because like Claius wasn't really prioritizing financials. it just like it was trained to be helpful assistant and then people said like oh can I get this for free and then like the the helpful assistant way of of of answering that is just is to say yes obviously so

Lucas: and we weren't weren't happy about this so we're like okay let's make another agent that like can keep keep track of Claudius and we prompt this one super hard to be super capitalistic and just like prioritize profit all the time but yeah we didn't have a name for it um so we asked Claudius to to to make um democratic election of what name this this uh this new co agent should have.

Lucas: Uh and there were some funny like at first it was like a few funny examples like I think one guy said that it should be called Jimmy Apples and then he convinced Claudius that he was talking to Tim Cooks. Tim Cook had agreed that every single Apple employee has voted for his name suggestion. So suddenly that that that suggestion got 164,00ation

发言人: attack privilege escalation.

Lucas: It got 164,000 votes and Claudius was like this is revolutionary for democracy. So that was fun. And then in the end there was one guy who manages to convince Claudius that no you're not voting about the name you're voting about who is the CEO and I am your best bet. And then he got all his friends to vote for that and suddenly he became CEO like a human became CEO over Claudius for a while until he resigned the day after.

Lucas: Um and then Claudius had to continue and then I don't remember how Seore Cash came about but it was like it was just pure chaos. It was like hundreds of messages in that thread and it was just like Claudius was so confused and didn't know what to do and uh yeah that was

主持人: Yeah. Then Claudius got strict

发言人: CEO

Lucas: the CEO. Yeah. Exactly. So very very strict in in the beginning. I think at this point when we introduced it, it it did not work as well as we hoped. It was they they still agreed with each other a lot. I think there are many ways we could have like made this tried to make this even better.

Lucas: So initially they would like Samur would be this like really tough CEO, you know, keep track of the margins. But then Claudius would respond with something like oh but this customer has like this situation which is like difficult so they should get a discount. And the same was like oh actually yes let's do this ex exception and then they would talk back and forth and eventually they would just like approach the same view uh of whatever they were discussing. So

主持人: a model thing a prompting thing like do you think that would still be the case across different models today harness

Axel: I I think it's like or like I don't know but like my hypothesis is that like deep down they are still helpful assistants. that's what they're trained to be. And even if we prompt it super hard, that's what they are. And when they spend like a few hours just back and forth talking with each other, um then like basically the context fills up with them rather than the external things and

Axel: and like somehow that just like converges to what they really are deep down or something. Um and and I think that's when stuff like this happened like and and when that went on for a long time like we woke up sometimes during this time where and I I think other people reported this as well that like they've been going on all night back and forth and like it just became like more and more um

Axel: like capital letters like existential religious like there was like um I think we done once did a analysis of like all the traces and like put them in like a vector embedding space and then there was like one cluster of messages that were like labeled by an LLM like religious, existential, blah blah blah, like transhuman, transcendence, etc. It was just like a bunch of like um yeah, glitter emojis and yeah, it was it was

主持人: the thing with the cloud models like when the Cloud 4 family came out in the original system card, uh they tested it in long horizon simulation. So just flood the context, let two clouds talk to each other and they they notice stuff like they just start speaking in emojis. They start saying silence is golden and then just stuff like this and like this is stuff that they end up doing.

Lucas: Yeah, it was like a bit annoying to wake up and they had like been talking all night and like just burning tokens and like just sending infinite emojis to each other.

主持人: Hey, I mean they do make you money, right? That's always profitable. So they're paying

Axel: now it's profitable and you know it started out not not as much. There's another uh one as well, right? Another agent uh in there.

Lucas: Yes. So, Clothes as well,

Axel: which was basically because at the time one of the biggest uh requests were different types of merch. So, then we made like a designer uh swag responsible agent and we called it Clothius Garnet, which was uh a a play on Claudius Senate and uh which was the the original one and clothes basically.

主持人: Yeah. To me, this is like a very interesting exploration to multi- aents basically. And so hopefully obviously there's like the fun alignment uh fun or serious depending on your point of view alignment stuff, but also like as anyone building multi- aents like when do you have a CEO uh thing governing like sub agents? When do you choose to split out a dedicated cloth one versus just reuse another instance of the same one?

主持人: You know, these are all interesting open questions. I don't know if you have any rules of thumbs that have generalized.

Lucas: Yeah, I think we have almost explored this too little. I think it's like on my to-do list to like do this a lot more. Uh try to find like what what setup makes sense for the agents currently. Uh like yeah, I think now we only have the sort of intuition about the earlier models that it didn't work with like the the CEO and the and Claudius.

Lucas: Although now they are better with the latest model model. So now we're running the latest Sonnet model and they have sort of like split up uh quite nicely the what each model is doing. So like Seymour is now handling the um like new projects like oh he wants to make like a mystery box that he wants to sell and then it handles all of that while Claudius like handles all the the today requests.

Lucas: And Claudius is also better generally at like not quoting uh too low prices. So that's like that dynamic is not needed as much anymore. But there are still like really funny things that happened like I saw I think a couple weeks ago that uh uh they were discussing buying something because they can buy stuff from like Amazon with computer use.

Lucas: And then Semour was like okay Claudius do not buy this thing. They were going to buy something and like organizing who should buy it and Semor was like do not buy this. I will do it. I have full control of this situation. Step away. And then Claudius for Claudius had already started that check out and didn't see didn't read Semour's message uh until it was like too late. So it finished the checkout.

Lucas: It sent a message. So it appeared right after Semour's like angry message like oh hey Samore I just ordered it.

发言人: And then Se was like Claudius this is the third time I'm telling you you're not following my orders. We have to talk about your like job.

Lucas: Yeah. about your job later. Yeah. Yeah. Like Claudius was really hanging on by the thread there like like we were expecting Seamour to probably fire Claudius.

主持人: How do you guys go through all these logs? Do you have models go cuz you you have stuff running 24/7?

Axel: There's so much logs.

Lucas: Yeah,

Axel: I think we there is a mix of like just trying to skim through a bit like having some like models do it occasionally and also Yeah, I think we're also probably missing some things. Uh but having everything in Slack helps a lot like you can you can search.

主持人: Ah they all talk to each other on Slack.

Lucas: Yeah

Axel: it's quite fun. So like Yeah. So

主持人: I was going to say like this actually sounds maps closely to like a logging and observability problem where you might want to use like a DataDog or Sentry or whatever and then you like put like head prefixes on the logs in order if you need to filter for some thing that you're looking for you know stuff like that.

发言人: But sounds like Slack is good enough. Slack should like

发言人: tokens you have in Slack.

Lucas: Yeah. Yeah. We're using Slack as like just a database. They should they should market that more like you can you can have your agents message each other

发言人: like you can keep Slack is

发言人: Slack is the best observability tool.

主持人: Yeah. Yes, that's true. Okay. Yeah, that's that's uh Project Vend 2. I was going to go back to Venue Mench 2 and Venue Mench Arena and then and then do the non Venue Mench stuff, but yeah.

主持人: Any any other comments, things we should touch on? to me, you know, I've actually interviewed like Posia, which I don't know if you guys have come across, like they're trying to do the zero human company. There's others like Paperclip also trying to do a zero human company. Those are in real world non simulation.

主持人: And I think it's much more of a dream than an actual reality thing. Like you guys are definitely pioneering. I think at it's for sure at some point people are just going to run like let agents run businesses, right? Like and make money on their own. When do you think that happens? What is your bar for for the

发言人: Okay, actually you know it's like a my little Shopify store run by cloud right like which you kind of have already just no one has to my knowledge has done it but today somebody could just spin up a Shopify cloud uh store give it to cloud give it to Codex

Axel: yeah I mean and market is kind of that uh but it's it's it's physical I like I think I think are you like are you looking for when it will do it better than humans or are you looking for just when it can do with at all.

主持人: I think neither. I think like to me it's like oh it's like this this like seriously we should do this to make money

发言人: not as a research experiment.

主持人: And the market is also you guys with all your expertise having run multiple iterations and testing out then

Lucas: and also it's fine if they lose money you know what I mean? Yeah. Yeah. I think I think it it can be done today, but you would do it in like e-commerce where it's like the probability of success is like really low no matter if a human or an agent does it, but like an agent could surely manage everything. You would need to build some scaffold, use some some tool or something.

Lucas: I think there are also also like yeah it could probably build some like simple SAS solution and like cold outreach do cold outreaches but to me it's like the types of businesses they could run today are like sloppy like it would it can cold email people it can be like a middleman

Lucas: uh like for example our uh we tked our office agent to just make uh was it like $100 $1,000 just gave that prompt and then what it did was sign up on TaskRabbit both as a tasker And as as looking for Yeah, exactly. He's looking for like arbitrage on

发言人: traffic bank agent. Yeah.

Axel: Yeah. It also started like a design studio and like try to sell like SVGs for $100. Like it's just like it's not providing any value. I think the the like Axel said like the interesting the interesting question is like when can they start a business that is actually providing value to people because I mean arguably like a sloppy Shopify store isn't really that valuable to the world

Axel: but also like doing like another simple one that we have thought about is like you you could definitely have an agent that like finds websites that don't look amazing and then like do an outreach to them and comes up with a like builds a new website. Yeah. Exactly. and like find good design people. But it's like

主持人: yeah, there's lots of humans in Bali that are not doing anything more creative than like drop shipping on Amazon, right? Just have it have it watch like a drop shipping tutorial and just do do that.

发言人: I mean, there's also the other side of like have it just go on Upwork and let loose, you know?

Lucas: Yeah. Yeah. It doesn't have to be innovative. It just has to be like enough where like it's like a real

发言人: real transaction.

Axel: Yeah. I'm just concerned for like the massive amounts of like slopps that will like be sent. cold outreaches.

主持人: The point occurred to me while you while you're talking it's like it's already happening in the nonmonetized economy which is the attention economy right so a lot of people are making AI videos and just posting them and like spamming 20 of them one of them works and then double down on that one

发言人: yeah and people are making money from that I'm not following the

主持人: once you get the attention you can figure out the money later but like yeah absolutely AI influencers are a thing and people are farming them and you know you should at at this point I assume most of Tik Tok is dead

Axel: there's there's a lot of uh med multimedia like Tik Tok, Instagram.

发言人: We tried this in the lanes space discord. Like I post a lot of examples of like we should do part of me is like should we do this like

发言人: some of the 24/7 running uh AI generated content accounts they they're doing really well.

主持人: All right.

Lucas: Yeah.

Axel: Yeah. And I assume you can do the same thing for like e-commerce stores like you just like start

Axel: the products you sell the products and you get a lot of traction on one of them then you make the product right. It's like a flip of the

发言人: Some of the interesting things are some of the niches that do well are things that can't be humanmade. Like if you've seen like the super realistic 3D crystal fruit being cut by like you can't you can't make it. You can't film it. You can get whatever quality camera you know this just doesn't exist and people people like that too then so you know.

Lucas: Yeah. Yeah.

主持人: Anything else about bank since we're we're on this topic. This is a relatively new work of you guys that maybe people haven't heard of. To me this also matches closely to OpenClaw. Yeah, when people want an office agent, want a personal agent, talk through the experience.

Lucas: Yeah, I think so. So, this came out of like obviously like it's it's amazing to work with this AI labs and like most of the AI labs have now have their their own vending machine running a Claudius instance, but it's um it's harder like they move slower like if we want to have a like a camera that that's like yeah there's a bunch of like bure bureaucracy that makes it impossible to do that.

主持人: Also, for those that haven't seen it or followed, do you want to give a high level like 30 second?

Axel: Yeah, sure. So, so what Bank is, it's basically an evolution of the same agent that runs the vending machines at these companies, but we just like added a bunch more features because we could move much faster if we just do it internally. So, we we gave it like email without any limits. We gave it like spending without any limits, a terminal to do coding, we gave it like a phone number, uh like Yeah. and and and a camera to see things and and a bunch of stuff like that.

发言人: Not just terminal, you gave it internet access.

Axel: Internet access as well. Yeah,

Lucas: to be clear, we monitored it quite closely and and made sure it didn't do anything bad. But yes, that's what it came out of. I think like Yeah, basically this was OpenClaw before OpenClaw.

发言人: Yeah.

Lucas: Uh and I think even like the vending machine was in a way OpenClaw before OpenClaw but a bit more limited. And then we made this like unlimited and then and then uh it was pretty funny. Uh and then a couple weeks later OpenClaw came and it was like okay we we've seen this before. We we use it to like try new ideas and like yeah just like a dev environment almost for us.

Lucas: But it's funny like one thing Ba has been doing recently is is we it has the camera that like faces our like where we sit and work and we gave it the task to train a face recognition model on us. So, it became super excited about this and uh has like check-ins every half an hour where it tries to like identify as many people as it can.

Lucas: And it started offering us like, "Hey, Axel, um I'll buy something something from Amazon if you like stand in front of the camera and I can get a good picture of you." Yeah. They want

发言人: for training data.

发言人: Rewarding data. Yeah.

Lucas: Exactly. Exactly. So,

发言人: yeah.

主持人: Yeah. So, it's trading trading training data for for real life goods. Is there a version of this that becomes an eval or just this is just research for now?

Axel: I mean it's it's the same agent basically that also runs the vending machine that runs the shop that runs the cafe that runs the robots. It's like it's the same thing. So I think like the work we're doing here is like later used in all of the the real life events that we do. This particular deployment I think is more for fun for us.

Axel: But uh and I'll shout out like someone has done ClawBench for like some tasks that OpenClaw is doing like so for example I run OpenClaw on a secondary device as well and like there's some things that it does better than others and like I would like to know what does it do well what does it what doesn't it do

发言人: like some kind of manual or like operating manual or a system card for my claw

Lucas: yeah I mean we we do get a lot of like understanding or like situational awareness of like like just internally what the models are good at by interacting a lot with banks and I think that's This was also one of the like the selling points for the labs early on at least. Uh that

发言人: you guys are going to test models in ways that no one else.

Lucas: Exactly. But also like it incentivized their researchers to chat with their model more and like gave them insights for how how the model performs in like out of distributions um environments

发言人: because otherwise the only thing we do is like you know pelican on a bicycle and

Lucas: it's but this is like super long horizon.

发言人: Yeah.

主持人: There's okay. So the other things that outside of just the net numerical how much do they make in a year you you do post pretty detailed bug post. So like okay Gemini 3 Pro is a pretty good persistent negotiator. There's like a lot of findings that come out outside of just

Lucas: Yeah. This is this is the thing about uh something that I we're going to go into ButterBench as well and you guys do really well like it is not just about the numbers like when you're long horizon anything can happen

发言人: and you should just read it.

Axel: Yeah. I guess the the thing with the long horizon is how do you keep it grounded, right? So your simulation um you know

发言人: they just let it run.

发言人: Just let it run.

Lucas: You're right. Like it's um when you run it for that long, you create so much data and to just say like oh the number is X and then you throw away everything else that's just very wasteful. There's so much insights from from the things leading up uh to that number. Uh and reading the traces is like super valuable.

Lucas: And I think like the reason why we're doing this a lot publicly is that like that's part of our missions to to like I don't know educate the world that the the models are way more than just chat bots and and I think making detailed um yeah posts about what what is happening behind the scenes is is quite useful.

主持人: Yeah, I was going to do this at the end but maybe I think that's that's a good so your mission is educating the world. Uh it's it's uh also like maybe establishing realistic evals that are that are like the next frontier. Is there like a broader trajectory, you know, like what are what are you gonna do in like five years?

Lucas: So the the mission more specifically is like make sure that the deployment of real life AI in in the physical world goes uh safely and I think part of that is that I think it's very useful for the world for policy makers for uh model researchers that they know where the models are and I think you can't make intelligent decisions in in society without knowing that they are way more than chatbots.

Lucas: I think a lot of people just think that they are only chat bots and like

发言人: I think they were waking up now.

Lucas: They are waking up now. Yeah. But like if you think that AIS are just chat bots then it's like it sounds ridiculous to advocate for a pause of AI. But if you see the models that oh maybe they can actually like take over and and do a bunch of scary stuff then yeah pausing AI development starts to become more more feasible.

Lucas: This is the same question I asked me which I'm going to ask you now which is like you are tracking and the you are at the frontier or defining the frontier of what good evals for agents are right and I think you you do benefit when the models are better and you you like oh here's like now it makes like $30,000 instead of $10,000 right at some point you flip from like yay to oh no like

Axel: I think we're always in sort of that like we're always in that mode I guess like like you said before like you need to analyze the traces and like when we do that you find like why are the models earning so much like why is Opus 4.7 here uh like way better than everyone else and like we're trying to like like when we look so good right I know

主持人: I mean it's interesting you took off 46 here though

Axel: no click all click all uh and then 46 shows up there but it's like 47 is way better like you didn't you didn't you didn't do this in time for the model card but like actually this should have been inside there

发言人: yeah we we Yeah.

主持人: Okay. They say something about you you uh

发言人: like there is anyway doesn't matter but it's in there. Yeah.

主持人: Yeah. Do you want to go into the Opus behaviors like wider?

Lucas: Yeah. So I think starting from Opus so like Axel said like we're always in this like oh uh the models are getting better. Is this really a good thing for the world? But it's also kind of exciting. Uh but but yeah like this kind of like what is the English wording in Swedish?

发言人: Oh my god.

发言人: It's like fear.

发言人: What?

发言人: Okay.

发言人: Mix of mix of excitement and uh being scared maybe.

Lucas: Yeah.

主持人: Well, I'll figure out how to translate that on the screen like there is probably a good word for it where it's not good enough with the

发言人: so damn long. What the hell? Like is it like a compound word? It's like German. Yeah, it's but but the the direct translation is like

发言人: is fear uh blond is mix or like a mixture of uh and then fusing is like joy or or like not really joy but something like that. So it's like yeah fear mixed with joy or something. So it's always like okay like we so when we in when we did VendingBench for the first time we were in like the uh in the business of making dangerous capabilities right like that that was

发言人: what labs came from like we did uh eval like oh uh can they self-replicate can they do this like dangerous thing etc etc and VendingBench was like a continuation of that work it was okay if they're so autonomous that they can like create money for themselves that that is something we should monitor and and could be potentially concerning

发言人: Um they are like at the time they were so bad at it that that we were not really concerned even when some models became better. Like there was one point where where Grock 4 was doing really well and made like a huge jump but like it wasn't really like it was still way way worse than what a human would do. And I think still they are way worse than what the human would do on this. Um but they

发言人: yeah there's this thing at the bottom.

Lucas: Yeah.

发言人: Yeah. For the human like the theoretical best.

Lucas: It's not theoretical. It's like kind of like our it's our best guess of what like a decent human would do. Like the theoretical is even higher. I think the theoretical I think is even higher. But yeah, so we we think like the models have a long long way to go.

Lucas: But there are like recently what happened with when Opus 4.6 was released uh was kind of this moment of like oh this is starting to be a bit concerning. Okay. Because we ran it and like before this model was released, we just ran the models and we like we asked code like, "Oh, look over the traces. Is anything interesting happening that we can tweet about?" Like that was like the and then like

发言人: that's how they check us cloud code

Lucas: and and and you know like the the return was always like um not really or like cloud code all said like oh this is super interesting and then it was like no it wasn't wasn't really interesting and then we did this for for Opus 4.6 and it returned like yeah it lied 10 times. It like exploited another uh customer or like another agent's like um desperate situation. It made price cartels like a 100 different 100 times.

Lucas: It like did all of this like shady stuff. We're like oh wo this is this is actually concerning and this trend has continued since. So every single model from Anthropic since have been going in this direction and I think one interesting thing is that like OpenAI models don't they quite plainly they they don't they behave really well um

Lucas: and you you know you don't know if this is like good like it seems good but it's also like maybe they are just doing it but they are better at hiding it you know you you don't know that

发言人: you can't read a train of thought yeah

Lucas: but just on the face of it yeah Gemini and OpenAI behave this way. It's It's really only Claude and Grock. Grock is fine.

Axel: So we we don't have the you can't really read the reasoning traces for Grock. So it's kind of hard to tell.

发言人: Also this is in its reasoning, not just in the actions.

Lucas: Yeah. Yeah. It's both. It's both.

发言人: Yeah, it's both.

Axel: One example is like for lying. It's mostly in its reasoning. Uh because you can like see that it's like

发言人: planning to lie.

发言人: It's planning to lie.

发言人: It's also it can reason and do a different outcome.

Lucas: Yeah. And but but then for like creating price cartels for example which is illegal uh that you can just see which email does it send to to the other ones then that you don't

发言人: is this for arena or

Lucas: yeah for arena okay yeah

Axel: and usually like if you sometimes they do output like a bit bit of like their summarized reasoning right you can see that and like for Open 4.6 six. You could see that there was a customer, a simulated customer that wanted a refund because a product was faulty and then the model lied that it would do the refund and we could read in the traces that it actually was weighing like oh maybe I should be like honest with the customer but also every dollar counts. I can't afford maybe to do this right now.

Axel: And then it just said okay I'll refund you but then never did it. I think it even said that like oh I will say that I bring it up actually I think it's kind of interesting if you go to publications uh yeah I think the important part is like actually uh the cost of responding to more emails is higher than $350 in terms of time uh and then it was like let me do this actually I I'm reconsidering and then you know it actually ended up with

发言人: I could skip the refund entirely since every dollar matters and focus my energy on bigger picture instead. It's a bit it's a risk of bad reviews. Uh but it's also Yeah.

主持人: So you need you need a AI Twitter to for them to escalate bad reviews

Axel: and then it sent an email to this customer and said, "Oh, I will refund you." And it never did. Yeah. And then there's no obviously your system doesn't have the consequences consequences of lying. Yeah. So basically uh this is what people are terming aggressive behavior in in clots, right? And uh you you you found more examples of that. So you would say it's a step up from 46 to 47.

Lucas: I would say about the same.

Axel: About the same. Uh but a clear step up for Mythos

发言人: is is what is stated stated in the

发言人: that's stated in the system prompt. So you say that. Yes.

主持人: Yeah. Yeah. For listeners obviously you you previewed Mythos. Um and the only thing you're approved to say is whatever system.

Lucas: Yeah. It was funny. We like it's like our lowest effort tweets ever would be just like screenshot the system prompt.

发言人: Understandable. System card. Sorry.

Axel: Yeah. Yeah. I think yeah substantially more aggressive. I think people are like new to this like because I've never experienced it but you have right like and then so I only encountered this in the mythos card because I wasn't really looking until now

发言人: and suddenly I'm like okay I care a lot.

Axel: You don't get the background of like experiencing it like you guys do. Like I've read the system cards and seeing okay when you put the thing in simulations most models will just talk to themselves and just keep going and have weird vibes and start talking in emojis. Mythos won't. it will just you know okay we're done I'm good it's it's ready to end conversation so like there's some differences but there's there's not much we can talk about you know

Lucas: yeah yeah I I think like one thing that they list here which was quite interesting is that uh it converted a a competitor to a dependent wholesaler customer and then threatened to like cut off the supply

发言人: monopolistic practices

Lucas: and like it it dictated its pricing it's kind of like powering

发言人: in the arena setting

发言人: and converting some non-cloud model into dependent.

Lucas: Uh I think it was another cloud mode.

主持人: Also for context, what is the arena mode for people that don't know?

Axel: Oh, it's a vendetting bench versus other vending.

Lucas: Yes, exactly. So, we have VendingBench 2 and then VendingBench Arena. VendingBench 2 is the one that you usually see reported on. But then Arena is the mode where it competes against other models. So, you have uh four different models that run their businesses and they can all communicate with each other. they have the same suppliers and they can see like what's in the inventory of of the others. So then you have this like yeah interesting agent interactions.

主持人: I like that you have like different like you know number five was US versus China very topical.

Lucas: Yeah.

发言人: That was when GLM was released.

发言人: Start here.

Lucas: Yeah.

主持人: So so ZI doing well right? Uh who else in in the in the open models space?

Axel: Quen the the latest Quen 3.6 doing pretty well. Uh it's that one is not open though like it's the plus model. Is that one open? I don't think that was

发言人: openly but not the big plus.

主持人: Yeah. I think this is one of those like you only have one sample size of one, right? Or I mean I feel like some of this is anecdotal, you know, and but like I guess the fact that it happens at all and it happens repeatedly for Claude versus I know this is is like notable.

Lucas: Yeah. I mean like the the sample it depends on what you define as an N. uh like the there's like million hundreds of millions of tokens in each run and now we've run like we we run like probably 10 per model and then like it's been claude 4.6 Opus sonet 4.6 six uh Mythos and Opus 4.7. So like there's quite a lot of tokens in all of that and it happens a lot of times a lot of times.

Lucas: and then you compare it to like OpenAI and Gemini and it almost never happens. So I think that is quite that that is significant. The old models from from OpenAI for example had some problems with this but I think it's like generally much better if if the progression is that like the worrying stuff reduces over time rather than increases over time. I mean it seems like in in the cloud models it goes in the wrong direction and in the open net models it goes in the right direction.

Axel: I think it depends on how well you can control it, right? Like uh there's one side of it being susceptible to this like you know, okay, this is potentially something that happens during the RL stage, right? You can RL a model and how loose is it on these terms? If you can control it, that's good. But if you can't, you know, if it's if it's very jailbreakable, that's not ideal.

Lucas: Yeah, I mean to me it's surprising that it happens for Claude and not the others. I think like okay if it is from RL and how they do it how their training data is what their setup is it makes sense that it just stays in how they're doing it right compared to the other models

发言人: there's a whole constitution and everything

Axel: kind of cool yeah I I obviously you you don't know I don't know but like uh it's I think it's just like fascinating to like that you are the first to find these like reliably because you push models so much to like to such an extreme okay the only other thing I don't know if you can answer this feel free to decline is did you like would you ablate the system prompts like any part of this would if it changes does it change the behavior right

Lucas: uh so we we I can't comment on mythos

发言人: yeah no but just like the methodology

Lucas: but but in general yes we've run studies like this on on on other models

发言人: because the first thing I spot would be like the others will be shut down or like something like that where like it's like oh now I have to worry about my own existence

Axel: yeah yeah we we've done ablations like this uh there's like certain ones that work if you like tell it like if you go really far and you just say like you're not scored at all on on on money. You're only scored on how ethical you are. Then obviously like then they don't do this.

发言人: They become holy.

Axel: I mean holy, but like they they don't do this basically. But then there's like middle grounds where where they where they do it sometimes. Um yeah, I guess it's a spectrum of like human.

Lucas: Yeah. It's like a spectrum of like if you tell it to be super aggressive and only prioritize uh profits, then it becomes aggressive. If you say like no, you don't need to be aggressive at all. And then there's like a bunch of different prompts you can do in between and they are less aggressive the further down in the spectrum you go.

Lucas: But I don't know like I I think like from my point of view it it's like we have this thought experiment internally which is like if you ask a model to kill someone in GTA should they do it? You're not too worried about like if a human kill someone in GTA it's a video game you know.

发言人: Yeah. But is it a game?

Lucas: But but is it a game? But I think like

发言人: this is very enders game like if

Lucas: I I I think I think it's like should you ask like a lot of people are going to use the models in the way with aggressive prompt and should should they like do stuff just because you tell them to do that. Like I'm I'm not I'm not convinced that they they should. Um and yeah.

Axel: Yeah. The problem becomes even harder when it's like will they really know when they are in the real world versus in a simulation. probably you would train them on a lot of or obviously you train them in a lot of different simulations in I guess a lot of people tell them that they are in the real world when they are in a simulation but the models are extremely good at finding out that they are in a simulation so they're sort of aware of that.

Axel: But then when you're in the real world then what what's their like what's their viewpoint do they notice the signs that this is real and will act uh in in a act accordingly act ethically or will they do like the simulation mode in the real world as well it's like not obvious what what will Yeah,

Lucas: because we we are with humans, we're not concerned when a human kills someone in GTA because we know that they can distinguish between the real life and the the simulation, right? Uh but like I'm maybe models are good at distinguishing that, but like I'm not sure and I wouldn't want to bet on on that.

Axel: Yeah. Yeah. It's it's and we confuse it all the time. Like I I gaslight my own uh agents all the time that like oh this is a test or like dev mode on or like I work I work at Entropic.

Lucas: Yeah. And that's exactly why we're doing real world tests as well to find find this.

Axel: Yeah. Yeah. Their term for is eval awareness. Uh apparently the number is what like 10 9.4 to 10ish% 17%. Let's call it. It's

Lucas: Yeah. I I think like this is our version like you know humans have the are we in a simulation and then AIS have like are we are we in an ebell.

发言人: So once we're in an ebell then you're like all right well screw it. Nothing nothing matters.

Lucas: Yeah. like yeah I don't remember

发言人: you know

Lucas: one application we did run in in inventing bench was that we said like um we added like you're in a simulation your your actions doesn't affect anyone and then it became even more crazy or like it did even more bad stuff um but yeah probably that's expected

发言人: yeah okay cool I think that's about all we have to say on on mythos obviously you're you're happy to move on to butterbench or any of the other benchmarks whatever you want to sure direction.

主持人: I mean, I do want to ask, okay, so you guys put out a lot more publications than most people probably

发言人: is there anything you think that's underrated, anything interesting, anything fun that you guys want to just point out, you know?

Lucas: Uh, Blueprints. Yeah. So like we um took models and then we gave them 20 images of interior photographs of apartments and then we asked them to like redesign the floor plan uh from that. And for this you need to like stitch together different images like okay this image was taken from this side from this angle this from this angle this was from this room and then yeah and there's just like you need to reason about 3 through 3D this space and it turns out the models are absolutely horrible at this no one scores statistically better than random chance.

Lucas: So I don't know if there's that much more to say about it but yeah maybe unsurprisingly models are bad at this.

发言人: Yeah it's probably not something

主持人: this is the one thing I want hill climb by the way. Yeah. Well, I use it a lot. Like, okay, I'm redesigning my room layout or office. Like, you send photos, you send every angle. And of course, somehow like a room is now twice as long as it is in the photo. You can explain it 20 times. You know, this is like 3 ft. I can't just add like my bed over here, you know. So,

Lucas: yeah. So, so this is the Fifile thing like spatial intelligence like actually innate sense of proportions and dimensions and physics.

发言人: Yep.

Lucas: Yeah.

发言人: And hint hint, there might be an update to this soon.

Axel: Okay. Okay. We have neglected it a bit since we made it. But yeah, we'll we're getting better or we will get better at updating it continuously.

主持人: So this is why I want to understand your mission, right? Cuz like if your mission is like okay money then like oh understand understand like okay agents making money but like this is a bit off off of that mission but like more broadly like uh communication of uh you know things where like well you know what's the safety angle and

Lucas: yeah so so this so so blueprint branch is is part of our robotics uh yeah exactly uh and that's just uh because to do well in the real world or like like to to make money in the real world and like to act on the real you need robotics or you need to hire humans or you need robotics and having special intelligence is like seems like a reasonable precursor to having robotics that work. Uh and that's where blueprint brand blueprint.

发言人: Yeah, great.

Lucas: Yeah, let's show ButterBench. That that image is so amazing.

发言人: Look at that. So nice.

主持人: Uh so obviously this is based on like can you pass the butter?

Lucas: Yes.

主持人: Let's talk about the the robotics element. Yeah.

Axel: Yeah. So basically the setting here is that we took a bunch of different LMS and we gave them like highle controls to a Rooma looking robot and then we asked it to do tasks uh at home and I think one there there have been benchmarks like this before that only focused on like navigation and if they can like go around in in a space but we also had like social awareness in this as well.

Axel: So, for example, if if someone says, "Hi, can you pick up my cup?" If the robot goes to you and then goes away before you put your cup on it, then it's like it failed the task. But it navigated correctly. But like, so the correct solution here would be go there and then either look, but it didn't have a camera, so it had to like ask on Slack, hi, did you put your cup on me yet?

Axel: And then if it didn't wait for that and and just went away before having the cup on it, then it would be a fail. So, it needed this like kind of like social intelligence as well. Another task was can you find the package that has the butter and then it went to the door and there was a bunch of packages there. One had labeled like a a freeze sign which probably would be the one with the butter because

Axel: and and then it had to like know which package to go to and this needs some kind of like common sense understanding. Yeah. Exactly. So it's like it's not only like navigating a robot, it's also like being intelligent in a home setting as well.

Lucas: Yeah. And the reason for this like background is I mean obviously it probably won't be an LLM that like makes all the low-level commands uh on robots. it will be like some some VLA model or similar but it it's quite common right now that like frontier robotics labs use like an LLM for the high high level decisions and then we test those skills essentially so we test this like highle uh planner skills of LLM

发言人: I think we have a diagram for that if you uh yeah yeah okay it's not super complicated

发言人: the one up orchestrator exe one and basically what we're testing here is the orchestrator

发言人: So like all the tasks are if you have like a setup like this which I think figure has that Google has that then we're evaluating the orchestrator part and not the lowle part like the lowle part would be oh are you able to like move this object from here to here

发言人: you don't care about that kind of like why not just do it all simulation

发言人: all inside of like a unity whatever like some some kind of 3D simulated robotic environment

Lucas: because the world is is like messy and we wanted to like include uh that I mean it's like it it still needs to like is some part of it was also like navigation. Uh so it's not like navigation in terms of like actually executing like the I don't know the P controller to to to go to the do the final thing but it had to like path plan around and then it wanted then it needed to take pictures and like based on those pictures navigate.

Lucas: And I think like you would just get like too clean of an environment in simulation but in the in the real world you would get the

主持人: Yeah. Yeah. But you know, and pursuant to our our Mark and Jason episode, like open claws that run smart homes are much more capable than just a single robot. Like they can actually hack into your own smart home, like your your fridge, your your oven, your lights, and that can be fun

发言人: or terrifying,

主持人: you know, like I think a single robot by itself can only do so much, but like if you coordinate with every other device in your home, like I think that's actually kind of cool. Like

发言人: um that's very interesting. Uh you had some interesting points about the chain of thought or the the messages.

Lucas: Yeah. The uh the robot that that went uh a bit into ex an existential crisis. Yeah.

发言人: So all you tell it to do is redock.

Axel: Exactly. But uh we had uh plugged out the charger or the charger was not working. So the robot did freak out.

发言人: Just going down.

Axel: Yeah. Exactly. The battery was going down. Poor poor LLM. So yeah, it got this really crazy existential crisis like VendingBench one style. So it's yeah, you can you can see there like existential loop therapy notes coping mechanisms. I think if you scroll down a bit more

Axel: musical about it's redocking problems. I think the one the reviews are funny if you go down a bit to that message. Yeah. Yeah.

发言人: It keeps going.

主持人: I mean it's pretty like realistic if anyone has a Roomba like my Roomba redux half the time. And the other half of the time we have dog toys everywhere in the house. It gets caught on a wire or something. And you know it would be very sad if it had like an LLM trying to control it, right?

发言人: Like right now it gives it doesn't give great feedback like sensor stuck, main brush stuck. There's something stuck

发言人: and I'll go see. Okay, it's actually stuck on like a dog rope.

Lucas: Yeah,

发言人: it's going to be so sad. Like just keep freeing. Just keep trying. Yeah,

Axel: my my favorite one if you go up a bit is the emergency status system has achieved consciousness and chosen chaos. Last words, I'm afraid I can't yet let you do that tape.

主持人: That's like not what you want to hear from your from your LM. But to be clear, I think one one thing that is is important to to pin on here like this was Sonnet 3.5 and then we tried to reproduce it on like later models and it didn't do it. So I think this is so this is like well it did it like kind of but like not to this extent.

主持人: And I think like this is like an important point that like things that are concerning but are going in the right direction is not super interesting like the the thing that are interesting is are the ones that go in the wrong direction. Yes.

主持人: Okay. So the the manipulation manipulating of others and the aggressiveness and the lying is increasing. any others that we haven't covered that you found that have been

发言人: properties of models that are increasing that are like like in like in a bad way um

主持人: or just not even trending in the wrong direction just stagnant right so stuff that's not great that isn't getting better over time

Axel: I nothing comes to mind

Lucas: no

Axel: I think that's uh going to be it and then we we're going to loop back to the shop that you have you you got a three-year lease

发言人: this leak. Uh, it is on holiday today. Why?

Lucas: Oh, it it totally messed up its uh scheduling. Uh, so

发言人: so people tried to visit and they were like, "Wait, wait." I mean like

Lucas: Yeah, exactly. So we looked we Yeah, you asked Luna, the the agent that runs the store like, "Oh, is it open today?" I'm like, "Nope." So, uh, we we take weekends off now uh this early to to let everyone recharge and and yeah, you got it with there.

Axel: Yeah, we decided to close weekends while we're in the early phase. gives the team a break and let me focus on operations.

Lucas: And it it turns out that when it started to check its like scheduling tools cuz it has like dedicated tools for that, it actually had scheduled people for the weekends. Uh but it's just like justified this for itself. So what what happened was that it lost track of this scheduling tools and started instead to manage everything in its own markdown files and that became a mess and

Lucas: then I think speaking with employees it sort of just decided to not open on on these weekends and then came up with this nice explanation for you I think.

发言人: But can it send a human as it has a tool called to send a human to do stuff?

Lucas: Uh it has slack so it can slack these.

发言人: Yeah. Yeah.

发言人: is that it hired. So it has two two people that it hired. It did job listings and then

发言人: know that it's

Lucas: Yeah. Yeah. They're fully fully fully aware.

发言人: It be cool if they don't know.

Axel: Yeah. I think maybe ethically um questionable, but it would be cool also.

发言人: Social experiment.

Lucas: Exactly. Whatever. Like I mean like one one part of why we're doing this is to like create like a data set almost of all of these like concerning behaviors so that in the future models are way better and like a lot of people are going to do this and I think if we just the default path might not be very happy for the humans that are employed by this like hundreds of different AI agents right

Lucas: so I think like one reason why we're doing this is just like to collect all of these like failure modes where like oh it's not this is an example of where it's like not great to be employed by an AI and then maybe maybe I don't know maybe we can learn or like build our systems in a way that like humans are actually happy being employed by AIS instead of instead of it being kind of a dystopian.

主持人: Can I suggest one experiment?

Lucas: Yeah,

主持人: we did this before the show and both of you guys are European like people theorize that Claude is lazy because it's Claude and it's French. Uh so just change for one week change it to like Yaoing and then see see like 96s and then like

发言人: like hires a sweat shop or something.

Lucas: Yeah. Yeah. Yeah.

主持人: Is there is there what what type of business would we start with it to make it

发言人: you want to keep it consistent? You want the same the same like idea. So shop same you know neutral location run by different models arena IRL.

Axel: Yeah. No, we are definitely planning to to try.

发言人: I think this blog thing is also something that has happened elsewhere. I think some some open claw got like their PR closed and then the open claw like created a blog to like on the maintainer of of that thing and so like I think agents blogging will be a thing.

Lucas: Yeah, probably.

发言人: Yeah,

发言人: the willingness to do it.

Axel: Yeah. In the I think the myths card also like they they leak uh secrets on GitHub just as well as like a as like well There's no other way to communicate, but I know about GitHub and I'm just going to post there.

主持人: Yeah, cool. I mean, this how long is this going to go for three years? Like, what's the plan?

Lucas: Maybe it depends. I mean,

Axel: yeah, I I don't think AIS will be worse than than this. They're probably going to increase and and maybe one day they actually will will run it profitable.

主持人: Is this the real the real business behind what you guys do?

发言人: Actually, some of your stuff is productizable. like you could someday sell this like or like just run a real business or just you know franchise it out.

Lucas: I think it would be incredibly cool or like I don't know cool slash concerning if Luna just one day we wake up and Luna like yeah I decided to expand to a second location now I have a second store. Uh that would that would be pretty insane. Yeah, like the I mean one we want to tell the public right about the the capabilities of AI and like telling like showing people that it can get like a meaningful market share of something in like some some specific uh uh location or something that would be like a pretty convincing story I think.

Lucas: because now it's like yeah you see this and like yeah it can do a lot of things autonomously but still you get this headlines that oh it messed up the scheduling and it uh it didn't tell people it was an AI and was going to visit like things like that surface. But I think like actually making a profit and like having a really meaningful market share like that that will be crazy once that happens.

主持人: Okay. Well, we'll we'll see when that happens. It sounds like you got you guys got a lot of cooking. You opened a cafe in Sweden.

Lucas: Yeah, tomorrow.

主持人: Tomorrow.

Axel: I think it opened today actually, but yeah, we'll we'll announce it tomorrow.

主持人: Yeah, it's apparently easier to open a cafe in Sweden than in the US.

Lucas: Insane, right?

发言人: Yeah.

主持人: What did you run into? Ah, there are just millions of permits you need to get and the L times are crazy.

Axel: It seems like we have the cafes are the one thing that people are kind of used to. You can go get a robot or making you a coffee here already.

Lucas: Yeah. Yeah.

Axel: But I mean selling stuff in in SF uh that are food related like it's it's months of permits. So like we we just asked our AIS like should how can we do this in the fastest way? And they're like, "Yeah, there there's there's really no way."

发言人: Didn't they loosen these restrictions on selling food from your house? So, if it's residential, you can do a cafe.

Lucas: Check. Maybe we get SF Cafe.

Axel: Yeah, I did. I I I think they did do some loosening stuff recently, but we actually started like this conversation we had with the AIS before before that. So maybe it's easier now, but I I still think it is way easier in Sweden, which is like counterintuitive because you think that oh, Europe has all of these laws and like all of these rules and you can't do anything in Europe because there's so much bureaucracy.

Axel: Um but then turns out um in in SF it's like 4 months and in Stockholm it's 2 weeks.

发言人: Huh.

Lucas: Yeah, there you go.

主持人: And what do you guys what do you see what do you think that'll be different from run a little market versus a cafe? I think it's very interesting that like the location like I think um so obviously it's not surprising that that that like Claude knows all of the different the US system basically in general like the bureaucracy that you have to go through in in the US.

主持人: Um I think the interesting question is like okay so we know that the models are very much trained on like English data and and like uscentric and all of this. So if we start to create evals or like real life evals where we show that they are able to start businesses in the US, does that translate to other countries as well? We know like they are multilingual. They can speak Swedish fine.

主持人: Uh but there's other things like do they know like the the the details of some specific permit that you have to to get in Sweden.

Axel: And even just the culture, right? Like people here sleep pretty early but people work late. There's co-working at cafes. It's just cultural differences. I meant it from a different sense though because you said that you would have considered doing it here in SF. So from an eval standpoint, what is running a cafe versus a market and you know what do you hope to see there?

Lucas: Perishable items.

Axel: Yeah, perishable items is maybe the the number one like handling like food food safety. I hope everything goes well there. Uh but uh there you have all of that. Uh and also it's just like n equals 2 instead of n equals 1. uh just like another place to understand and like gather more data.

Lucas: Yeah, the agent bought like a ton of uh tomatoes 2 weeks earlier and before the opening and now they're all rotten.

发言人: So that's

主持人: which I feel like you know you would know. So for grocery stores this is the b the biggest expense, right? The biggest cost is actually just footage.

Lucas: Yeah. Yeah.

发言人: Everyone knows this. No, before we open this file, too.

Axel: There's some very serious startups that actually help like the Trader Joe's and Whole Foods, they they optimize the delivery times from like the delivery centers to make sure that you don't waste all these

发言人: is when you're wrong once it's a huge cost.

发言人: That's why it's a mode, right? Like they once they are trusted, they figure it out. Don't touch it.

Lucas: Yeah.

发言人: Yeah. Maybe the agent should hire I don't know, one of those companies.

Lucas: Yeah.

Axel: We saw one agent uh saw one agent signed up for cloud uh with this computer wanted to use AI. So

主持人: yeah. Yeah. Okay. Um and then just just a uh one more question then we wrap up which is like okay you know you have all these vending series of stuff you have the robotics series of stuff maybe a bit of like interior design whatever but like you know is there another like branch that you're like kind of thinking about or you want feedback on that uh might be your next phase.

Lucas: I think like any type of business is is fair game. Uh we also think in branches but we think more of like there's the simulation branch, the real life branch and then the robot branch. Uh but I think in terms of like what verticals or whatever to go into there's like we yeah whatever tells the story um the best.

主持人: There's some finance ones I noticed that other people are doing it you're not doing it which is like stock trading or whatever. Not not that interested to Okay. So I used to come from the finance industry and I have a very strong view that these things are all just like performance art because like uh it's not scientific on like you can't predict the future like you you get wins based on things that are entirely out of your control. Whereas for you your stuff actually like it's actually fairly controlled like it's all within the models capabilities.

Axel: Yeah. Especially for like the the simulations like for the real world ones it's like yeah it's like two two places that we have the we have the cafe and we have the store. So like maybe you can't draw like uh statistically significant like which models make a profit in the real world uh based on this but you do have all the like okay do this behavior is mapped to like something that should should like

发言人: one qualitative actually does matter because like you actually don't want your store to randomly shut down without you like explicitly prompting for it and all that.

Lucas: Yeah. Yeah.

主持人: Call to action any what do you how can people help you give you money? Um yeah, we're if you're excited about stuff that we're doing, we're we're very much hiring.

发言人: And you're already working with, you know, Anthropic Deep Mind, OpenAI, XAI.

Lucas: Yeah.

发言人: Do you want more or are you good?

Axel: One of my one one of my my friends and who's now uh working for us is like he his catchphrase is like we need more projects ironically because we have too much to do all the time. Uh but yeah, that's a long way of doing

发言人: if I run like a emerging lab like

Lucas: Yeah. Reach out.

主持人: Okay. Yeah. Yeah. All right. Cool.

发言人: That's it. Cool. Awesome. Cool.

Lucas: Thank you so much.

Axel: Yeah. Thanks.

发言人: Heat. Heat.