递归决策模型与系统一：无需大语言模型处理每个决策的智能体架构

Okay, so maybe some quick definitions what what does that mean?

So system one models the naming of system one models from Type Safe when they released Jev not even two months ago now I think came from this concept that Daniel Kahneman popularized in Thinking, Fast and Slow. This idea of like the system one part of our brains is making very fast and intuitive like judgment calls, gut reactions, reflexes. Maybe maybe reflexes more purely neurological, but that idea of like a lot of the decisions that we make are not actually like purely rational. And so system one is fast, system two is slow, more deliberate like logical reasoning, problem solving, etc.

And I think what was a little unclear when they when they announced Jev was whether they intended system one to be like their proprietary name family of models or like that's they were expecting all the other people to call them system one models.

I would say most most people are calling them decision models. And and so I wasn't really willing to to wait for the people to decide. And so so when I came up with recursive decision models you know it was just it was just kind of using that. So arose by any other name tomato tomato we don't really need too hung up on this idea of what are they? Are they decision models or not?

Ali here argues that Jeff makes the judgments, not the decisions. Your code makes the decision. I think that's that's absolutely correct.

And yet I think that that much of the zeitgeist and much of the folks working in the space outside of type safe are are kind of rallying around this idea of decision model being the term. There is the holdout camp of refusing to call them anything other than zero shot classifiers.

Anyways, I don't have a dog in the race. The naming just trying to explain what I am talking about. And I think that one of the things that is now connected here to this idea of system one, system two, Danny Kahneman, all this stuff maybe related to how we behave, but I think much more interesting from this perspective of building AI systems is what if reasoning was actually many small decisions?

And yeah, maybe a different way of seeing this like what what if behaviors or tasks that we might think of as requiring a lot of reasoning or a lot of like deliberate cognitive processing could actually be solved in a different way with a lot of very tiny, very, very fast decisions. We're going to keep coming back to this question throughout the talk and I think it's it's really key in what's interesting here. So

Um, and then like many others, I was also chomping at the bit to um, I think it's actually champing at the bit. Anyways, I was champing and chomping uh, to get off the freaking Jev weight list, which thankfully got off real quick.

Um, they did a good job with that. But I think that one of the things that I've been sharing with people is I think that part of what TypeSafe did so well and what captured the attention of so many engineers, AI engineers, is um, this idea of Jev as convergence, right? Like decisions converge.

And that is in direct contrast to generative AI, which has been you know, blowing up like crazy. It's arguably like the fastest growing um, you know, market in the history of the world. And um, so much of the attention and money in in the space has been about generation, right? And that's awesome and there's amazing things that are possible with generation.

But if you really think about it, like even if you if you have infinite tokens, right? You kind of have infinite divergence, right? Like the generative stuff, it's actually hard to um, converge. It's like we still have slop. Is someone still needs to decide like what's going to make it into production. And maybe you've got your like code factory set up there. All those like agents are doing those decisions for you.

Um, but at the end of the day like the the important thing that I got me excited and I think I think this is why many other people are so excited about about Jev, these system one models, these decision models is [snorts] this this idea of yin and yang, right? It's a counterbalancing force.

Um, and so the decision model cannot generate text. It can only uh, uh and pick and choose amongst options. Um and that is like in direct contrast to LLMs which have this ability to just like spit out uh free-wheeling open-ended um text. And in the early days actually hard to get them to structure their output. So I I see this as as yin and yang.

Um I am not going to go super deep into um recursive language models. I would highly encourage you to check out my talk from the summer from AI uh Engineer World's Fair this summer, recursive coding agents. It's on their YouTube channel. It's also at Um my slides here are also a um a website and I'm going to make them available publicly after I record this. Um that is going to go a lot deeper into why I think LLMs are the new reasoning models, uh why this is a really important paradigm.

Um but we also have to think about composing these together, right? And the um I am going to show some examples as pure decisions all the way down. That's very important.

Um this bottom left of what I'm I'm calling decision as a tool. Um so you could you can also this is what my recursive coding agent talk was about. It's like replace RLM with like agent, right? Coding agent. Okay, you got the root coding agent, you got the sub agent, da da da da da.

So I would say like this decisions as a tool or Jev as a tool was is like the obvious one. And within you know 24 hours of people getting access to Jev, there's tons and tons of examples here, right? People using Jev for compaction, people using Jev as the thing that decides whether a tool call can execute or not. People using Jev as a router to decide which um which model is going to get this particular request, right? So that stuff's um it's all very cool. There's some really interesting cases there and people just went buck wild going super fast um figuring that

I think to me what's I'm more interested in is um is not as obvious how you would set it up. Uh and that's where the root is a decision model and you only can you could choose as a decision to call to a language model or an agent um as one as one possibility. So almost everything that I'm going to be talking about with RDM is going to um be in this paradigm where we're actually letting the root or the parent of this system be the system one model, the decision model.

can pose their answers into the player's next action and keep repeating that loop as the game changes. This is what we call a composition of AI primitives.

of AI primitives.

Now this composition handles the moment-to-moment play at a second

basically the first thing that they're showing is this harness that they built of decisions to handle the like the first person shooting aspect. Is it going to hold, carry on, walk, dodge, blah blah blah. That has these actions, right? This is where I it's so simple and it's very much in line with this idea of composing systems together, recursive decision models, right? A graph of different types of decisions.

Um here we go. So this is the second composition.

that decides where to explore next and they can work together to make it through a level.

Okay. So now you have two compositions. One is the first person perspective. One is this like bird's eye or like map aware perspective and by composing those two things together you have a doom playing decision agent.

And now to me, I don't know, maybe I missed something, but I would say like um I have seen almost nothing. Very very very few um examples that are actually like taking this forward, that are that are moving this. This is a super powerful idea of composing these two systems together. It's really insane that with like essentially two separate um but coupled um so two distinct but coupled decision agents, right? You now have with pure Jev calls um a doom playing agent, right? Like that is really wild and I think really underappreciated despite the fact that probably like I don't know, millions or tens of millions of people saw this video. And it comes back to this question, what if reasoning uh was actually many small decisions, right?

And um or what are the types of activities or categories of things that we might naively assume would need reasoning when in fact what we need is a smart way of architecting small fast decisions. Okay.

Uh while I was developing these um this idea, this concept, I've I've given myself three rules. Okay? The three rules of RDMs are

one that the context stays outside the model. And so the code um can cut it into slices. So the the in the Doom example the the full state of um the game uh is outside then those two little sub harnesses, the first person shooting harness has some specific subset of that. The um the map exploring birds-eye view decision model has another uh view of that.

Uh the second thing is this uh and now this is where it's a little different from that demo is the decision can choose to call another decision. And again up to this maximum depth we talked about this recursive tree. Now these two things are the most important fundamental aspects of RLM as well, right? The context is externalized. The agent can make sub agents. The RLM can call sub RLMs and sub LLMs, right? Um the so those two things are like copied straight from the RLM idea and then it's applying that directly to to system one models, decision models.

Uh the third piece is first it must work with no generative model at all. And this is very important to me um because I again I'm like all this like Jev as a tool for a normal coding agent is is is cool and there's some really great examples, but it's it's obvious to me. What was really interesting in terms of novel, what might unlock some novel behaviors, some novel capabilities um is what if there's no generative model at all. Also proves that the that the RDM architecture is actually uh correct. Like it should be able to work with no generation. And at the same time it must compose with LLMs and with agents, right? Um so those are my three rules in developing this concept of recursive decision models.

So first I want to show the the decisions all the way down example. Okay? So decisions all the way down.

Um you might be familiar with this idea of a Galton board. Um maybe you learned about this in the context of learning about um the the Gaussian distribution or or um uh what do they call it? Normal distribution. So like each ball has like a 50/50 chance at each pin of um falling and then you build this like bell curve um Gaussian distribution. So the simplest way that I could think of to to visualize this idea of um like recursive layers of decisions is now what if the pins could decide? So you could think of this kind of like a binary search like so each um numbered ball just like oh is this one going to go higher or lower, higher or lower? Um and so this pin is like splitting them in half, this pin spitting them in like to its half, etc. Um and again this is kind of like a silly, simple example, uh but now when those are smart, then we separate them perfectly.

Now, system one model like Jev, decision model, whatever we want to call them, uh well, we can ask about we can ask many questions at once. We don't have to just ask one on one axis. We could ask in multiple dimensions. So, we can now visualize that in 3D as like the ball's falling, they're being sorted uh by color in one axis, by number in another. We could put the cross-sectional views here and see that in the case that this is a perfect um classifier, right? Um it's perfectly classifying the numbered bucket that it should go in in the end, the uh colored bucket. So, again, this is very much a toy example in the sense that like actually what you you could do all of this in one classifier call, you could do all of this in one decision call. You don't need um this like recursive tree or like iterative tree to do it. But I'm um I want you to keep the visualization of this because it's exactly what's going to happen in some of these more specific examples of recursive decision models. Okay. So,

hierarchical search was one of the launch day uh cookbooks of Type Safe. And they what they did is they took um this Shopify uh database of 12,000 product categories and they implemented like a beam search algorithm with hierarchical calls to um Jev to figure out from from a user query which product category um the user's query was describing.

And what is unique as a simple demonstration of what's possible with recursive decision models is you can do the beam search without the beam. So, essentially at each layer here, we're simply asking well, um, of the next sublayer of products, what is the next one that we're interested [snorts] um, in exploring? And so, uh, in this in this example, we're exploring we're not exploring all 12,000 categories at once. That would like overload the context window. We're taking the parent category. We're deciding, okay, the user said laptop backpack. Okay, which category am I going to explore next? And and iterating along uh, from there.

Um, and we could decide in the code how we implement it. Do we do greedy? Do we pick like only the top one, right? Do we explore [sighs] top three um, that are rated uh, with the different probabilities, etcetera? But you could kind of again, it's like similar to a binary search algorithm in theme, except that like the compositional unit of it is a classifier, is a judgment, is a decision model, is a system one model. Again, too many names. I don't want to fight about the names.

Okay, visually, um, we can show this um, as well. And so the uh, Jev quickly became uh, multimodal uh, just a few weeks ago, about a month after the initial uh, launch. And um, Lily over here uh, posted like the first variation of this, which is like wherever you are getting a hit, you are um, now then kind of like zooming in to have like a feedback loop and like figure out you know, ask again on a on a smaller box. Okay, is this really uh, in her her case looking for um, beetles, right? Um, and then Nathan from the Type Safe team kind of followed up on this. And um, what's interesting about his implementation is it's much more like um, this um, binary search in the sense it's like a recursive halving of the of the image, right? Like a like a golden golden spiral style like halving, right? And then within those blocks, within those buckets, like what what is it that we're seeing?

So again, this is this is the these are simple simple examples that you you might not need recursion here like a you know a good vision model could probably answer the question in one shot and I think some people have have pointed that out clearly, but I'm I'm trying to give a number of different visual examples of of how a recursive decision model could work and where different ways of you seeing what that might look like in terms of the recursion.

So we've got all of these these lines that are making our steel or whatever. I'm I'm not very good at Factorio so apologize. I actually would like to my goal actually with this is to have my recursive decision model teach me how to play Factorio cuz I never really played before, but it's a it's a pretty cool like very open-ended way of evaluating the systems, the metric in systems. So you can see here it's got all these lines. It's figuring okay this group is the the one that's got a problem. This line is the one that's got a problem. It's this element, the inserter that I think I need to go fix and I need to replace it. So now it's going to run over and go do that.

And now the thing that I I want to point out here is there's there's no reason here. There's no large language model here. There's no thing generative here, right? So it's kind it I don't know. Maybe it's a little bit of a toy example and that it's literally like a matrix of these things and they're all the same. Um, but then it moves on to the next one. You'll see actually I should have maybe added some pathfinding composition of uh recursion because it ran like the long way around to go the next one. That was not totally perfect, right? But this is just an example of what what if you just looked at this without knowing it, I think you would I don't know. At least I would assume that you need some kind of reasoning model here. You need um and um yeah, you like this is like I don't know, like Cloud plays Pokémon level uh thing and it to me it's just absolutely fascinating that this can be done uh without any uh generative language model in there at all.

So again, we come back to this idea. What if reasoning or tasks that we might have assumed naively required reasoning can actually be done with these many many many small decisions?

All right. So the the weakness or the downside, um the limitation of decisions all the way down, right? Is that something has to have made those decisions. Like what what um are the allowed decisions? And um for any of you who have a type-safe account, you might remember the the quiz. You have to pass a quiz to get an API key. And the question is, can Jeff generate text or not? And the answer is no. And if you put yes, they won't give you an API key. So so the um you something like programmatically um whatever like a script has to do it, another agent has to do it, the application itself has to have pre-enumerated all of it, right? Um, if you can only have decisions, then there's no generative aspect.

Um, and what's crazy to me is that for games, for browsers, and I think even for a lot of elements of computer use, that actually might be enough, right? And I think if you kind of visualize now back all these examples that we just showed, uh, you hopefully can understand why with a tree of many, many, many small decisions, we can actually um, find things. So you imagine a game, the number the total possible um, actions available to a user, the total um, possible state of the game um, can be like enumerated programmatically and recursively addressed to make judgments, make decisions about what what the next move is um, in a browser. So these slides are is actually a website. Um, in this particular website, you can go up, you can go down, you can start a video. So there's like three interactive elements in the DOM. Um, the the you think about just think about it on your computer. Like really if you're on your computer right now, like how many um, how many actions are available to you on the current screen uh, or with like the hotkeys? Like it's it's a lot. And if and typically computer use has been this benchmark where you think of, oh, only the smartest frontier reasoning models can actually figure this out. Um, well, maybe not. Like maybe actually if you could just enumerate like all of the possible options to you um, you can navigate it with uh, very quickly um, and and we're going to get towards real-time machine learning examples in a little bit. Okay.

Um and what's very interesting about then when you can um now have a generative element which I'm showing as like the square here. So like in this previous one it's like the generative ones diverge, the um the decision models converge, generative ones diverge, decision models converge. That's kind of an animation. Similar idea here. Now you can start to have So these are actual real call trees from the demo I'm about to show you, the smart house. You have these much more organic shapes and these much more like emergent graphs. And I think there's this um I have this idea that like that pairing, the yin and yang of the decision model and the generative model itself is kind of this very powerful automaton, right? That can kind of ratchet through the system and and generate emergent behavior complex systems uh from the pairing of these things. And you can get a some sense of that from from the the different geometries of these graphs that you're looking at without seeing the actual behavior.

Um Type Safe had a really cool uh cookbook uh that they also launched uh with or maybe it was called a um I don't know, demo or whatever um of a of a smart home. And so the the key thing from the smart home demo was that that they showed was when the problem cannot be solved purely with a decision, then it can go to an LLM to do generative element. So what I've done is adapt the smart home demo, make it much many more rooms in the home, many more appliances and things like that to show how the recursive decision model can can interact with the home. You can express a very high level thing like we're off to bed and then you can ask that and it will do all of the substeps very very quickly to get you to door locked and lamped on lamp on, everything off.

And then you can have these more like oblique type things that it's learning how to do over time. Let's watch a film. And then we will ask now it's going to like work through all those things.

In order for the decision model and the language model to talk to each other, we need some kind of shared interface. And so this is what I'm calling the map. And so again, the decision model can only point, it can judge various things that were given to it. And LLMs in their sort of most raw form have no structure to their output. It's perfectly open-ended stream of tokens, right? And so what we need is to kind of have a shared interface. And so I I call this the map.

And the the map is you could think of it as a a a set of like pointers in the sense that it could literally have like a name like TV on that has a very simple description and then some function in this case is like flipping this boolean to be on, right? It might have a much complicated function underneath it, like this sort of like loop here. But the key idea is that the the decision model can decide from the map which things are the right thing. It doesn't it so it can decide lights on. The code underneath it can implement the light can implement the lambda function lights on. Okay?

The LLM now has a structured interface to add and edit and modify the map. So the LLM can basically be learning from new user behavior that okay, this movie time is now the combination of turning on the TV and turning off the lights. And so now there is a shared interface, the shared substrate that both can talk to. I'm not going to go into it here because it gets a little bit more complicated, but I also want to point out that the map itself can be multi-dimensional. So it can essentially be representing embeddings. And and for example, like Kieran Clausen from Every had a a really interesting idea of using using Jev to create low dimensional but highly semantically meaningful embeddings. So so this map that I built into the RDM codebase can can handle that as well.

And here's why this is important, which is Jev is fast and cheap. There's decision models that can run on your phone, on a crappy 10-year-old laptop, on obviously you can get a better one on your new your new MacBook Pro, whatever. These things are very fast, They're very inexpensive, either from a compute perspective or a dollar's perspective. And that means that we can run many at once. We can we can actually speculatively run a bunch of them when uncertain, all in parallel.

And now the thing is that we need to if we just let all of those act on the state, um, we're going to have a lot of collisions. We're going to have a lot of challenge. So um, an example from how your CPU is working would be like that there can be many things reading in parallel, concurrently, um, but now we need some kind of lock when we write because we don't want to uh, collide, have colliding uh, writers. There's also an interesting idea also from Alex Tseng um, at MIT in Omar's group, the inventor of RLMs, about um, speculative tool calling, speculative programmatic tool calling for agents. Uh, and this is um, you know, informed my thinking here. But here's here's this idea applied to audio, which is like we have uh, I'm speaking uh, these sentences. We're having we're making decisions where I haven't even actually like finished the sentence. And so we essentially have um, at various times the arbiter is like deciding, okay, with the given information that I have across all of these different channels, what is the decision that I'm going to allow through? So you could you could imagine that like each one of these uh, lanes might like want to um, might be voting for like a certain action. And the arbiter's job is to figure out like where it has the confidence to actually like allow that action to change the state because again, if we let them all uh, act, we could have a lot of collisions. Let's say just a simple example, right? There's two there's two that are making uh, the their different models, they're making the same kind decision um and one of them uh goes a little bit faster, changes the state um before the second one makes a different decision that would have changed the state the other way. You can imagine all kinds of problems or collisions that could happen when you do that. So this this pattern of the arbiter is what I've developed um as a way of like um um it's like a gate for what can actually now um act on the system, change the state of the system. And it's very useful when you want to run many of these things in parallel.

Okay. So I'd love you to try it. Um my very first implementation of RDM was as a DSP Pi module. Um this is in part because the DSP uh team, in particular the the folks at Compound AI uh were very very quick to add support for System One API calls within um DSP. Um and secondly DSP is a very um opinionated way of working. It's very structured way of working and I found this to be very useful playground for um essentially ensuring that the ideas of recursive decision models um were structured in the right way that um the modules were behaving uh in the right way. Uh I have also uh made a standalone Rust package um RDM which is great when uh you want to bring uh very very fast and portable um RDMS to all kinds of systems. So yeah, check it out um in either my DSP fork uh if you're a DSP nerd or um the RDM package if you want to play around with Rust.

And what's really important to me um personally is this physical enmeshment that we have with with our devices, right? Um I got like tech neck, right? Like my head's always down. I've got this like crazy like shoulder problem and tension from mousing all the time. When I was in grad school with really stressful deadlines, I was having all kinds of um repetitive stress issues and I've got every crazy vertical mouse and trackball and ergonomic keyboard that you could possibly imagine. But the like there is great um power and economic benefit in working with computers, in working with devices, in working with um our phones and our tablets in terms of being interconnected and be able in terms of being able to just do our normal jobs. And obviously anyone who is an AI engineer is going to need a computer to do their job. And at the same time like there is just all of this um I would say unnecessary suffering that is caused by the coupling of our physicality, like the coupling of our physical bodies to the physical computer, like through the traditional interfaces that have made their way in as the popular ones of society, the QWERTY keyboard, the mouse, the touchscreen, etc.

Okay, so some of you might recommend recognize this um this sequence. Uh I used it in my Recursive Coding Agents talk and it's also at the back of a t-shirt uh from the AI engineer code event in New York a couple years ago. And so so here we go. We've got the evolution of of the AI engineer. Uh you know typing on the laptop not great posture. You know now here it's got like the AI sparkle features. So we're we're vibe coding here maybe. Um you know in the third one you can see there's no more computer anymore. So now we're just like talking to this like orb. Um that's pretty cool. And uh then in the this ultimate like state of enlightenment, right? We just like think and and um things are manifested uh in in our world or I I don't even know what kind of uh wavelengths are coming out of this guy's head.

So I'm going here. I'd like to get to the the right um making some progress but think right now with uh with operating system one and my recursive decision models I'm I'm much closer to to the audio piece. Um and so I want to um I want to sort of take a um to go back in time a little bit. Um this these videos are um I'm getting old. They're they're coming up on uh not quite 20 years old. It's like 17 18 year old uh videos and I want to show you some experiments um in what I call radical human computer interfacing um from some of my work and music. I'm just going to show these really quick and I'll explain a little bit what's going on. So

so over here [music] so so just watch here really set this up. All right. So we've got like uh hitting these four different locations on the laptop and that is uh being classified uh to make a drum kit, right? Drum on by hitting our laptop. So, the laptops used to have um accelerometers built into them back when we're still uh have disk spinning disk hard drives. Um this one I'm not going to play um so much of, but you can um uh on the left one I'm like triggering the drum kit here. You can use the tilt of the um laptop to control different effects. I'm using the hit of this thing as the side chain compression. Um so it's like a ducking compressor every time I hit it. Uh and that's as much as I'm going to do on that, so I'm not going to go on that.

But this is um I want to bring this up um because it's personally meaningful to me. It's also the way that I actually learned um how to program and um the way that I learned machine learning. And this is through the Princeton Laptop Orchestra. Um it just celebrated its 20th anniversary uh this spring. And really the idea of the Princeton Laptop Orchestra was what would what does it mean um to have the laptop itself be the instrument? So, there were pieces where you're performing where the trackpad was the instrument. There were pieces where the keyboard was the instrument. There were pieces where all the computers were networked together on um Wi-Fi and like syncing super super um quick like low latency clock times to send messages back and forth to each other.

And um I was able to work with this uh brand new cutting-edge real-time machine learning framework called the Wekinator developed by Rebecca Fiebrink um to actually develop um and again this is like this is like 19 years ago, 18 years ago. Um these um very very small um you know K-Nearest Neighbors um like single layer neural nets um you know things that look much more like principal component analysis than like what we think as like transformers, large language models. And and um and you can actually um do uh what you might call like on the fly um machine learning to to um classify and map and do non-linear interpolations. Um it was just so so much fun. So I've been having a lot of fun with coming back to um that a little bit and kind of merging um my excitement about RLMs, um my excitement about Jev, these decision models, this this invention of recursive decision models and now towards um this kind of personal goal of just like like I want to be able to get things done through my computer and uh work in this field without hurting, without like touching a keyboard all day, without touching a mouse all day.

Um and then uh while that's happening there is a very small um uh based in Chuck there's a new thing called Chai that's like Chuck AI that's sort of the successor to the Wekinator that I mentioned before. Very very small model model operating on a very very small slice of audio, 40 milliseconds, 80 milliseconds, 100 milliseconds, right? And essentially trying to just map from some very crude binning of so the features are just like some bins of audio, frequency bins of audio, um which is the right one. Now this is supervised, right? So it it's it knows nothing in the beginning and we can't trust it in the beginning.

So there is one layer on which there's like a teacher model that's teaching this small real-time model, right? And then there's a second layer on which the system is independently figuring out when it actually trusts the small model enough to follow it. So we're going to watch it now and what you're going to see is in the beginning there's a big delay between the word and this comes from Philip Glass's um uh knee play one part of uh Einstein on the Beach. And um so you're going to see that [music] it's going look up here for the one two three four. So there's a big delay right now. [music]

Okay. Now it's it's supervised learning. So this [music] early numbers where it's hearing them more one two three four are going to learn a little faster. So now see how it's speeding [music] up a little bit for the early ones but there's a lag [music] by the end of this thing you're going to see that we got down to essentially 45 millisecond um [music] time. Here we go. One two three four one two three four five six. [music] So now for those we're really locked in, right?

Oh and there's a little blip but it's learning and it's going. [music] So this is recursive decision model deciding um when to trust this smaller supervised [music] model that is learning on the fly and operating on tens of milliseconds of information [music] on the pure audio, right? So here we go. By the end it's really going [music] to get

Anyways, to me that's super exciting. Um really really fun to like come back to these um these real-time on-the-fly machine learning algorithms, which is actually how I first learned um about machine learning and and I first learned programming through through music. So um

I also want to So that's the audio example. I also want to give a shout out to this project called Hunch. Uh Matt Mastracci, not sure exactly how to pronounce it. He sent something really crazy visually messing with just embedding Gemma 2 um and essentially having this thing play Pictionary. So he has this thing classifying your doodles um on your machine in Web GPU in like what look at these numbers like 150 milliseconds. So again, I want I I just want to give this a shout out as like I'm not the only one working in this space. Audio isn't the only um way in, but these these very very very small um embedding models, encoder models, classifier models, right? Like there's a whole resurgence of this and if you can uh structure them and architect them into a bigger space, you can do really really powerful things. Okay. So

with this this is kind of where I 'm going is like how can we decouple our bodies from our devices? What would it mean to work on a computer without touching it? What would it mean to work on a computer without being able to see it, right? And that's really what um I'm building up towards with this project that I have called Oso, oso.computer. It's short for Operating System 1. It's not quite ready um but that Earshot demo that I showed you where we're very quickly from a single uh word uh even before uh even at a speed that would be faster than we can actually transcribe what we're saying. We're able to react and respond um to to our voice. starting starting with voice as the first input.

Um and yeah, just like a little bit about also as like a teaser and I hope you'll join the waitlist and try it out when it's ready is like it is a computer-used agent, but it's built on recursive decision models. So, yes, there are uh regular agents and RLM agents um and you can actually um uh you can um sort of farm out jobs or um uh give them to like your normal coding agents like Claude Code and Codex. Um but it's it's decision first, right? And so you can also do things that are super fast uh without touching the keyboard, without touching the mouse.

Um it's recursively improving. It's constantly trying to learn um what the preferences are um of the user and it's also trying to essentially always like that um Earshot demo, the the Philip Glass demo, it's also trying to like farm out the decision to the the fastest, smallest um available model once it can build trust in that. So, that's this idea of the arbiter, right? It's like figuring out, okay, where have we seen this thing do well before and then now how much can I trust it over time? So, it's a it's an evolving system.

It is a tool to control your computer without touching it. Again, we're starting with voice. It's an experiment in what I consider radical human-computer interfacing. Let's like rethink everything. I think there's this really interesting thing about everyone going into the terminal um with Claude Code and I don't think it's actually anything um special about the terminal. That's why um that happened. The terminal is clearly not like the best interface for it. I think a big part of what happened over the last couple years of us all going into the terminal was really about uh just realizing the limitations of these default interfaces that we've been given, these application interfaces, these default visual um interfaces, touch screen interfaces, mice, keyboards, all this stuff. Um and then yeah, lastly it's just like um I'm having so much fun uh coming back to uh what I consider my roots in in real-time machine learning and and it's really fun to be building this this computer usage and OSO. So anyways, I hope you'll check it out. I'm really grateful for your attention. This is a long talk. I'm using the opportunity to do the virtual talk to to really go deep um and yeah, I hope you'll check out the recursive decision model repositories and I hope you'll sign up uh for this wait list when OSO is ready at oso.computer.

Thank you so much.