递归自我改进（RSI）：超越自动催化与全栈优化的闭环AI进化

Speaker: The fundamental idea here is to build AI that improves itself. And I think we're getting there.

Speaker: For the first time, we could swap the optimizer as the optimizee. And this leads to a reinvestment of cognitive improvement into itself.

Speaker: You can think of RSI at many different levels of the stack, prompts, harness, weights, and you can go all the way to basically chips. Our goal here is to optimize all the levels of the stack. Our approach to RSI is that we have a multi-agent system working together that's been recursively improved.

Host: I'm so impressed by your belief that like this idea of a model improving itself,

Speaker: but it's not a belief.

Host: It is a dramatic statement, which is I think it's the first time I've heard you say it.

Speaker: Because we have been so thorough, we have not released things until we're super sure it's correct.

Host: Do you envision a system where the amount of energy that goes into it somehow decreases over time relative to the improvement? We can't share too much, but one of the very important things that we've tried to do. Welcome both of you for joining. We're going to be talking about everybody's favorite topic, which is RSI, recursive self-improvement. Maybe to start, it'd be great to just introduce yourselves and your backgrounds. Then we'll talk a little bit about what you're working on now, and then we'll dig into the content.

Speaker: Yeah, I think of myself as like a physicist of some kind. I studied that. Then I did the PhD in reinforcement learning at the MIT Media Lab. I really like building things and I'm very excited to be building a cool startup with Owen and Grace. My name is Owen. I was first studying math and then one day I realized that all mathematicians are proving theorems and I was taking a class called formal language and logic and they were showing how you can build a theorem prover and I was like oh that would be cool I can just build a machine that prove theorems for me that's how I got into AI

Host: amazing

Speaker: and then for my PhD I worked on continue learning and metal learning then worked for a few years at deep mind at a research scientist now I'm working on RSI I see a lot of pitches come in and a lot of them use the term RSI and I'll be like, "Oh, so what do you do?" And they'll say, "We use AI to create AI. That sounds very interesting. So what do you do?" We use AI to create GPU kernels.

Host: Which that doesn't to me kind of match the definition of what I would consider to be RSI. So maybe can we start by you defining what you think or what you would include in RSI

Host: definitionally?

Speaker: Yeah, I think RSI is definitely an overloaded term today. Many people when they say RSI they mean different things. So I think the important thing when you are differentiating them is to ask the question what is the optimizer and what is the optimizee like when you're self-improving who is improving right so

Speaker: there are people who saying okay we're doing RSI we have human using AI to develop better AI

Speaker: but that to us is just autocatalysis

Speaker: in a sense that the improver is still human with tools even though these tools are getting better and better but the idea and the improvement come from human brains,

Host: right? And just I don't mean to interrupt, but to be clear, most technical epochs have had autocatalytic effects, right? Like there's no way you can create a computer chip without a computer, right? You could argue that the internet development was sped up by the fact that people that weren't geographically located could coordinate, right? I mean, the steam engine, you could build better steam engines because you could get coal faster, etc. So, this notion of like using the thing to build the thing faster is not new. And so you're saying something different.

Speaker: Yes. I think for us in the history right over civilization we have this autocatalysis you know computer using computers to build better computers but the idea for this change for this improvement come from human brain but that human brain was never the improver never the optimizee right whereas when now for the first time we could swap the optimizer as the optimizee so the brain of the agents which is LLM the model can be the thing that you're optimizing and this leads to a compound this reinvestment of cognitive improvement into itself.

Host: But there has to be a reason that you decided to take on this hard problem as opposed to what anybody else would do which is like chew off pieces of the problem.

Speaker: Yeah. I mean I guess to answer before I answer your more important question here. The reason I did this is I think it's a hard problem and it's fun working on the hardest problem and I think

Host: but do you have an intuition that this is is the actual right answer as opposed to like something that's more pragmatic?

Speaker: Yeah, I think a pragmatic thing would be just from a control theory perspective. If you have a gap in the system, you can't have a closed loop and autocatalytic has no closed loop, right? So whatever you improve, there's still like a gap. humans have to come in. You have to pour in other resources to make it work. But if you have a closed loop, it you still have to pour in resources, but the marginal gains ideally get either constant or larger over time. And that's that's the fundamental difference. What's really different here can be explained pretty easily.

Host: So if you think of capability,

Speaker: yeah,

Speaker: what we're trying to do here is we're trying to improve the rate of improvement of capability, right? So like the first derivative of capability with respect to time is the thing that everybody is trying to improve. Yeah. Right. And now you can put that to be equal driven by two things. The nominator and the denominator. The nominator would be performance. Yeah.

Speaker: Which is what everybody is trying to improve. And then the denominator is some kind of cost. Let's say capital, right? Or difficulty. And this one is pretty is pretty set because the higher the capability, the higher the difficulty of the next step. Right now the numerator is the interesting part for things like steam engines and whatnot that was fixed.

Host: Yeah.

Speaker: Right. So then you end up with a very simple differential equation where you basically plateau.

Host: Yeah.

Speaker: Right. Now if you have a system where the improver the things being improved is also the thing that's doing the improvement

Speaker: you how you now have a new way to think about it and the the simplest case would be that the numerator is also correlated with capability which makes sense right because the better you are the better you are at also improving

Speaker: and if you do that the math cancels out and you basically get constant improvement over time that's that could one level of RSI, right? And then there's one more case which is

Speaker: now that the the numerator could be correlated with a higher power of capability. Yeah.

Speaker: And then you get exponential takeoff.

Host: Yeah. Can we dig in a little bit more between RSIs having closed the loop and actual convergence properties? Like for some reason when people see a a graph where there's like a a circle with an arrow to another circle then and it goes back and they look at that closed control system then they say takeoff [laughter]

Host: right which I mean if this is a nonlinear control system

Host: it's very difficult to know what the hell happens by the way most people don't seem to understand this distinction

Host: because we very quickly go to like takeoff and doom and all that stuff when we talk about these things. Well, as a network guy, you know that it depends on the exponent of, you know, your your function. That's self-referencing.

Speaker: And it's what we found is that it's almost not useful to speculate on what we think could happen because this is really an empirical problem. And the real way to know what's happening is to test things. and testing things is really really hard like reward hacking is one of you know many problems that we have to deal with but the the question you're asking is very empirical and for some

Host: interesting so but so I'm a networker that's how I would have answered I would have thought math people would have been like we've got this we've got this differential equation and I took out my pencil and I wrote down the answer

Speaker: I mean it is possible to write down the differential equation but you still have to empirically diff measure the the variable the exponent.

Host: So there's no analytical solution.

Speaker: I mean honestly we don't even have a learning theory of deep learning. We don't know like fundamentally why it works,

Speaker: right? And then now you're like five six layers on top of deep learning and so we don't have a good theory. I think the closest things we have are scaling laws and these are empirical laws. I think one misunderstanding people have about like our approach to RSI is that they think the entire system has to be fully automatic and digital. But actually if you look at our mathematical foundation we have a multi-agent system not just one digital agent. And in this multi-agent system human researchers and engineers are actually special case of these agents and it's this hybrid system of multiple agents working together that's been recursively improved.

Speaker: I see. And we don't want to leave out any component to not optimize them because like if it's possible to optimize in the model why wouldn't we do that because you will have more gains if you do include those. Anything that you don't optimize will later become your bottleneck.

Host: So one observation that we have made here is there are claims of speed is improving which is actually the case right? Right? So if you look at like the model release time, it is reducing. And so on one hand you can be like okay so clearly we're in some sort of a rapidly improving phase. But if you divide by the actual amount of capital raised by these companies that's no longer the case, right? And so you're like well maybe they just raise a whole bunch of money and then like AI allows you to apply that money. So, how would you, this is not for yourself, but how would you encourage people that are either evaluating claims about this or even working on them to think about not fooling yourself by throwing more money at the problem like you're actually showing that you're making an improvement here that's meaningful with the denominator.

Speaker: I think for us it's pretty straightforward. It's because if you get improvement and they just come at more and more resources being poured in, it's not RSI. You have to be able to back your claim of how much improvement you're getting.

Host: But do include like all the manhours and experimentation that went into that because I feel like at some level like they'll talk about like the actual flops that go in and money that go in but the reality is this is like the 23rd experiment and it's actually a human burning a lot of mental capital to get to that final configuration like do you actually include that in the denominator?

Speaker: I we do and I think it's it's good company discipline to do that.

Speaker: Yeah. And we've tried again to very intentionally from the beginning have a culture of tracking where things go cost time all that kind of stuff. I share a funny thing internally we have this thing people talk about leaderboards you know which is like score

Speaker: you know what I'm about to say we have something called a bleederboard

Speaker: where we keep track of the cost of experiments of everybody who's running experiments.

Host: That's awesome.

Speaker: you know and everybody it's the comp and right now people can train on single GPU within hours and

Host: Yeah.

Host: Yeah. But that's also totally separate, right? Like that's just hardware?

Speaker: No, no, it's the same GPU. It's the same hardware.

Speaker: using the same hardware and then you improve the techniques

Host: Yeah.

Host: Oh, interesting.

Speaker: Yeah. What has improved is like algorithmic techniques, data techniques, kernels, all of that kind of stuff.

Speaker: you know how much it cost to train GPT2?

Host: I think it's a few hundred.

Speaker: $100. I just Oh wow. So the same same GPU same GPU same data

Host: and this is just algorithmic and do those come from humans or agents

Speaker: a lot of them are auto research results

Host: but that but that's definitely the denominator that's that's cost.

Speaker: I think one of the most closely guarded secrets in the industry is how well they're doing on the denominator.

Host: Yeah. Because in some sense, even if all you were improving was your denominator, that would still be incredible.

Speaker: Right. Yeah. Of course. Right. And that means now you can sample that what what you can do at inference time.

Speaker: For sure improving the denominator is good. Yeah. For sure. Improving the numerator is good. Yeah,

Host: I've yet to see which is great that you have, you know, glimmers of both. They're complicated. this sound cool, they get you a good paper, but they don't actually hold up in practice.

Speaker: Yeah. And the reason is because this space is so large that it's easy to get lost. And the other reason is that often when you have this really complicated system and you hope that like this this this you know what is it the magic happens. Or you take the other way and you you kind of basically spend all your time trying everything.

Speaker: Yeah. And if you do that well, you can scale. And I think I think Richard Feynman said once that there are two kind of styles of science. So without a unified math framework, all these research are just randomly trying different things.

Host: Right. I see. I see.

Speaker: So so for example there are people who are doing recursive improvement of prompts, some on code. And our goal here is to optimize all the levels of the stack: prompts, harness, weights, chips. And if you think about it, there's not that many problems that you really need to solve. And one of those is the inference problem, right? because the more you can spend on the more compute you can give at inference time, the smarter the model becomes. and it's not just for technical reasons but also the most economically valuable thing.

Host: it kind of breaks the mind right so like just just for everyone listening. And again, that does not necessarily mean takeoff that can still be controlled. But it's just wild to think of this kind of compounding impact.

Speaker: Yeah. Yeah. We're a startup. We're not a research lab. And everybody wants to use the latest model for example

Host: just for the sake of discussion if if I if I wanted to make to improve the serving stack. You mean yeah in our

Speaker: well our system is not just using one LLM you have many different LLMs. Oh so you're different the inference side for all of them. Yeah, we try to be like as hands off as possible. Let it explore.

Host: Do you have like a single master prompt? Is it like improve yourself?

Speaker: Uh but even that master prompt that has been improving. [laughter]

Host: Really? It improves a master prompt.

Speaker: Yeah. And I mean and it's not one RSI system because you can define it mathematically. Mathematically we have defined it so that it's one system that can work on any problem domain that you point it to.

Host: and by everything mean any problem domain that you can point it to?

Speaker: Correct. But then there are physical limits for example and environmental constraints.

Host: Can we at least reduce bounds? So for example, do you envision a system where the amount of energy that goes into it somehow decreases over time relative to the improvement?

Speaker: So it takes less energy to get better results over time?

Host: Yeah. Do you think there's a universe in which that holds? but it just seems to disobey information theory and the laws of physics.

Speaker: Yeah. I mean I think at some point you're going to hit the asymptote so it will asymptote at some point but I think at least you know how many particles are in the universe.

Host: I'm so impressed by your your belief that like this idea of a model improving itself,

Speaker: but it's not a belief. No, it's not belief. It's empirical measurement. But also it's a question of what metric you use to view the progress. It's not a belief that this is happening. I think we're already seeing it.

Host: How do you have confidence in the results given you know like reward hacking where it did hack something and now you have this weird results, right? It's not easy. When dealing with nonlinear multi-dimensional systems, do you have confidence?

Speaker: absolutely. meaning like it'll get good and then it'll get bad, absolutely, and then on top of that you have noise. So what we find is that specific agent on a specific benchmark is not the whole story. So from that you could already see that intelligence is not a scalar. Exactly. So the digital part we could optimize and accelerate, but the physical part brings immense complexity.

Host: What I've yet to see is anybody put a balance on that, like to say in order for me to improve this by X to know the bounds.

Speaker: One of the internal jokes we have in the company is that Einstein would not have had these laws that tell you how to engineer GPS. And if some unfortunate engineer had to design the GPS system with general relativity, they would have had to add some fudge factors. Exactly, and later people would discover the underlying laws. The complexity of the world is such that the real challenges probably exist in spaces that are not being applied to, like that joke that there is some workflow somewhere in a garden hose company in Wisconsin

Speaker: that if we solve it we get you know

Host: I love that. I just want to quickly summarize so you're going after RSI in your case this is not autocatalytic effects this is actual RSI where the system improves the full system and for you it's an actual system which includes the harness and the weights. If you think about the problem is you've got a numerator and a denominator. The numerator is like how good the system gets and the denominator is how much energy goes into it. You believe you're seeing glimmers of the sign where both improve at the same time using this type of approach. Is that correct?

Speaker: Yes.

Host: when can we expect to hear more from you?

Speaker: Very soon.

Host: Amazing. Yeah. Thanks so much both of you for joining.

Speaker: Yeah. Thank you.