Matt Turk: Hi, I'm Matt Turk. Welcome to the Matt podcast. It's been yet another extraordinary last few days in AI with OpenAI, DeepMind, Anthropic, cracking some of the most famous long unsolved questions in mathematics known as the Erdos problems. A moment many view as a stunning breakthrough and yet another signal that AI is moving from doing the work we ask of it to autonomously making deep science discoveries. to unpack the moment and the fundamental advances in model reasoning that make it possible. I'm excited to welcome Dan Roberts, a top AI researcher at OpenAI, who comes from a deep background in theoretical physics and has a particular interest in the intersection of science and AI. In this conversation, we go deep on what reinforcement learning actually is, why it's the most important paradigm in AI right now, and what's ahead for AI and science. Please enjoy my conversation with Dan Roberts. Hey Dan, excited to do this. Thanks for taking the time.

Dan Roberts: Of course. Very happy to be here.

Matt Turk: You are the lead of the foundations of reinforcement learning team at OpenAI. So what what does that mean? What does the name mean?

Dan Roberts: The larger team that that we're on is called foundations and we think about reinforcement learning. So very boring foundations of reinforcement learning but the team comes from a mandate of of thinking about the science of reinforcement learning and a long time ago which in AI speak is like 6 months ago maybe a year I guess now two years. So before we released 01 and thinking reasoning models, we were studying this internally and and one of the advantages to being first or at least to being forced and and spending a lot of resources on scaling things up is that you can empower a group of people to not just work on making the thing work, but work on understanding how it works. And then beyond that, how do we scale? How should we think about scaling reinforcement learning versus scaling pre-training? So what what are scaling laws look like? But then going beyond that, what what sort of things does this kind of training teach us? What doesn't it teach us? We're very interested in at the frontier for exploratory scenarios. How do we either improve or understand better what reinforcement learning is doing? We have all this compute that famously we are in the process of of of of acquiring and we would like to turn that compute into intelligence and to do that we need to make thinking models and some somewhere along the way we interact with that process usually at the earlier stage uh for models you know not the next model but things that are like the next model or the next next model.

Matt Turk: Great. And uh quickly what was your path to open AI? So, how did you go from studying physics to being where you are today?

Dan Roberts: I did a PhD in in theoretical physics uh from from MIT thinking about the intersection of quantum gravity and quantum information. Thought a lot about black holes, quantum chaos kind of thing of what if you throw something into a black hole, what happens to the information? Does it does it come out? How if we think about black holes as computers, how fast are they? I was very interested in this fundamental question in theoretical physics which is how do you find a quantum theory of gravity. I also got very interested in this interest interplay between computation and the laws of physics. You know any computer exists in the universe in in you know behaves according to physical law. So the sort of computations you can do are bounded by the laws of physics and there's some sort of interesting relationship there. Black holes are pretty interesting because they sort of saturate some conjectured bounds around processing of of information.

Dan Roberts: And from there I did a posttock at the institute for advanced study and at around that time I'm pretty old now for at least for this field. So that was about 2016 uh was when the DQN Atari paper from deep mind happened in 2015 and then AlphaGo was in 2016. And I got very excited about these um about the possibility of uh machine learning and then and then deep learning was statistical science that lived in a similar framework to the sort of frameworks that we use to like study the rest of the universe. And so there's always this question of of of you know how does everything work? This three-year-old question of I'm curious about about everything. And if you look outward and you you care enough, you end up philosophy. Maybe if you're quantitative, you end up in in in in physics. Very very crude characterization. An AI that works is very fascinating or AI systems that work is are are fascinating because they are simple examples that do things that humans do. And then if it lives in the same framework that we use to understand everything else then you know it's sort of like you can you can draw the parallels between how does the universe work and how do I work or how does intelligence work.

Dan Roberts: So I got extremely interested in in AI and and deep learning. Then I uh went to fair Facebook's AI research lab at around 2017 2017 to basically try and use the tools from theoretical physics to understand deep learning. Deep learning was supposed to be this really difficult thing that you couldn't understand. And uh I thought maybe the tools of physics could could be helpful. Uh this actually culminated in a book that I wrote with a collaborator who now is um a collab still collaborator of mine now at at OpenAI working on the same thing as Shya. But we wrote this book the principles of deep learning theory that is a culmination of of of these sets of ideas of can we can we sort of use the statistical ideas of of of uh you know understanding statistical statistical systems like the gas in in in the room. we can characterize them with some simple laws of thermodynamics like the ideal gas law and and maybe we can make similar progress in understanding deep deep networks.

Dan Roberts: So um that was sort of my transition. I also had a startup along the way um and spent some time at Sequoia Capital uh as an entrepreneur and resident. So there's some tension between am I a scientist and am I entrepreneur but about 2 years ago after thinking about whether I want to start another AI company I realized that the the thing that was most exciting right now was what was happening at the frontier that there's some some some amazing scientific progress happening in in AI and to really get at the questions and understand what's going on you need to be there and you need to participate and and that that meant joining joining lab so I I joined OpenAI two two years ago.

Matt Turk: Great. Thank you for that. Where do you think we are in the evolution of AI being increasingly able to solve difficult scientific problems? I mean, certainly something that we've been talking as an industry about for a while now, but it seems to be accelerating perhaps just like everything else in in AI. But where do you think we are?

Dan Roberts: I think one of the interesting things is that this process is smooth. the there's no sharp point or I don't think there will be a sharp point where we'll say that systems didn't weren't able to be useful for scient the scientific process to their fullyfledged scientists. There will be sort of a gradual shift. If you had to point to one moment, maybe it would be the release of 01 and by OpenAI and and the sort of paradigm of test time compute and and and reasoning. But I'm sure if I tried to make that claim, you could go and look at GPD4 and and see that there's um glimpses of that sort of useful behavior for for the scientific process were were already present.

Dan Roberts: As a general point, you know, the the models are very good at certain types of things that clearly are amendable to to making progress in math. They're not open loop fullyfledged scientists in in any domain. Although, you know, neither am I. It seems like it's just this really nice gradual process. So it feels like a particularly fun week uh to be having this conversation because over the last few days uh there were a number of different announcements uh in the general field of AI and mathematics around the airdos problems. OpenAI came out first with this progress but like almost within a few hours Google deep mind had a claim as well on different problems. and anthropic had some claims. However, from what I understand, the open AI approach and the deep mind approach were were very different and that may be very interesting in terms of what that means for AI as a research scientist.

Dan Roberts: This conjecture everyone assumed was true and uh but could not prove it. One of the things that Chach GPT was able to do was uh assume it was false. And when you go against the grain and and do something contrarian like that, you really have to have strong conviction in what you're doing in order to persevere down a really long calculation path because there's a lot of choices that you can make along along the path. And if you get any of those choices wrong, if you if your ideas don't work, then you find out that that you didn't make any progress. And so um you need this really strong persistence.

Dan Roberts: And then you need expertise in this other field. uh which is like algebraic number theory, some sort of generalization of number theory on um you know things that sort of generalize the the integers and the real numbers. You go down that path really far, you can refute this conjecture. So that that was the big result. The the the big result was that this this conjecture of of this lower bound for the number of pairs that you can make is is false. Not only is it false, it was false due to a really interesting con connection to another field of of mathematics. And so you would have to be somebody who is aware of this problem as interesting which sounds like you know your expertise is one thing and then be an expertise in something else and then then also be super contrarian and go down this really long path and and then you could you would you would have identified the solution.

Matt Turk: The open AI approach and the deep mind approach were were very different. Do you want to compare and contrast the two approaches?

Dan Roberts: One of the approaches that GDM takes is to take problems, present them in a formal language called lean and then use methods to search for proofs in in that language. And some problems for problems to be representable, there's this process called auto formalization where you take English version of the problem and you translate it into rigorous formal statements and then you you conduct your proofs there. And it's it's it's designed so that the proofs can be airtight. No one has to go and check for for some hidden assumption or some weird thing or I guess it's usually hidden assumptions or definitions that are not airtight. But in that setting, which is a setting that DeepMind has has has cared a lot about, they were able to formalize some some problems and and use their use their system to prove them.

Dan Roberts: So that's that's one approach. Another approach is to just take the problem in English with mathematical expressions as well but just the English statement of it which is informal and understand what is meant by that and solve that in informal language presenting a proof much like the way a human mathematician would or a human mathematician who's not using lean and then you have to check it. It's the the verification problem is is is is harder because it's not something that auto checks.

Matt Turk: And that second approach was open AI.

Dan Roberts: Most of our results that we publicize as far as I I can think are all in the informal setting. We have we have language models that we've taught them to reason at test time and and one of the applications or benchmarks for that is is reasoning in in mathematics.

Matt Turk: Okay, great. All right. So let's get into reinforcement learning to make this uh broadly accessible. Let's start uh from the top. What what is the one two three sentences definition uh for reinforcement learning and perhaps give us a simple non-technical analogy for people to understand.

Dan Roberts: Maybe a simple thing to do would be to to give you two examples of of how you could try to learn something. you you as an individual. Um, and and maybe we can we can take a game or even a say say a video game, right? I I'm old enough that where I played the original 8-bit Mario Brothers, the Super Mario Brothers. And so here are two ways you could learn how to play. One way you could learn how to play is your dad takes it out um and plugs it it in and he boots up the game and then he plays for a few hours and then you you just watch him play. That's all you do. Um, so he's demonstrating how to play and then at the end of that he, you know, and he's not very nice, so he doesn't let you play and but then he like goes and runs outside and does something else and, you know, you sneak into his room. You you plug it in and you try to play. How how good are you going to be?

Dan Roberts: Reinforcement learning would be your dad's like here why don't you play? Maybe he shows you once or maybe he doesn't even need to show you because the the game is, you know, beautifully designed to to sort of take you from not knowing anything to to being able to play expertly as something called a curriculum. But you you play maybe the first thing you do is you run, you hit the first bad guy and you probably this example is dated, but you you know you you you lose a life, but then the second time you press a button and you jump. And so you're taking actions, there's an environment that's giving you feedback and you, you know, there's this this close connection between the environment, you know, between um actions that you can take and then the responses that you're getting. And and then the the final part is there's a reward. And the reward um you know, can can be something that you get pretty often for for instance, every time you do something, there's there's some score that goes up or it could be just something that you get at the end. So you play a game of uh of chess and and and at the very end you you get a reward which is you won or you lost but in the middle you don't really know how you're doing until until the very end. So this is called sparse resistance rewards. But I think this is a basic idea in that um and there's there's obvious lots of variance here um and ways to quibble with this but it's this this notion that you you interact with an environment you get an reward and often it's it's in a way where you get this sort of feedback as opposed to just trying to you know learn from data that you don't get to interact with.

Matt Turk: And why does it work and why is RL so powerful?

Dan Roberts: It works because of this ability to get feedback from the environment. you can go and learn, you know, if you if you're doing it right, you can figure out how to learn the things that that that you don't know. And I also think it's powerful because of this fact that it's it's much easier to learn when you're learning at at the right level for for you, right? So, if you want to learn, you know, addition, you shouldn't read a calculus textbook. You want to learn by being able to practice and and learn at at at the right level. I'm actually making the choices and and and learning from my own choices whether they work or not then then I'm able to like place it in a better context for for the set of things that I understand.

Matt Turk: Great. And then conversely what's the catch and how does RL break

Dan Roberts: the setting where very difficult is is the setting um that I alluded to before where you don't get much feedback from the environment. you have to take many many many many actions and then you get maybe yes that whole set of actions was good or no it was bad. For instance, you're playing a game of chess and you don't know what you know until you make all the moves that has an opponent so it's maybe complicated. Maybe it's you are trying to do a homework problem and it's a research level or you know like someone gives you a well- definfined problem like we give our language models and it's uh you know a problem that requires days and days of thinking there's so many choices that you can make along the way and at the end if you if you don't get any feedback at all if you're just hidden in the woods by yourself scribbling in notebooks it's very hard to make progress that way um because you don't have any feed you don't have any sense if you get a yes at the end or you get a no at the and you have no sense for which of the the actions that you took, which of the things you did were were good or bad.

Matt Turk: Okay, great. Now, let's talk about how uh RL has been applied in the context of large language models. So, was the first step historically RL HF?

Matt Turk: So do you want to define for people what uh RHF is and sort of how it works quickly?

Dan Roberts: The basic idea is that you could use uh collect data from from humans. That's so the RLHF is reinforcement learning from human feedback. So you collect data from humans and you train a value function. So you would show in the language model setting say two different completions from from from a language model. Ask them to say which is better. this this sort of comparisons could be used to train a value function and then you can um use that as a as a as a reward for for reinforcement learning process.

Matt Turk: Great. And uh you you do that initially with humans but then you build that into a reward model.

Dan Roberts: Yeah. So you would train a model for this and then and then now because during the training process you can't just pause your training run to ask some humans for for input. um right the the feedback the that that would have way too much latency. So instead you need a proxy for what a human would say. So you train this model based on based on the human preference data and then you can optimize against it or at least a little bit.

Matt Turk: One of the famous things in the history of RL is uh moves 37. How do you um train a model to encourage the model to do that kind of things and come up with brand new ways while being efficient and exploit known path?

Dan Roberts: Yeah. Uh so the great thing about Go is that you can just train it. It's a you know zero sum two-player game. You can train train in what's called selfplay. It plays itself and it can go from playing randomly to expert play and it will find whatever the the sort of best best strategies are. So if that means exploring great if that means exploiting uh actually I have a I have a funny story about this. Um so I I met Noam Brown in grad school. um he went to a different grad school than me, but he wanted to enter MIT's poker bot competition. And he had a poker bot that was the um best in the world, but it wasn't something that would compete against humans yet. He just won in this research competition. He collaborated with me and and another friend to uh enter MIT's Pokerbot competition. This was great actually for me because I learned some really exciting work in in AI and I got very excited about this while I was doing doing physics.

Dan Roberts: We were playing essentially this this kind of selfplay equil equilibrium strategy. There's some some nuances but you know essentially we could not lose assuming we did not have any bugs in in our code. Um the way this this thing worked was that it was a tournament where you would be paired with say another another person and play them in and if you you know depending on the amount of points you got in like some sort of roundroin setup they would eliminate the bottom half and they would keep going until you got to the final table which would just be say you versus the other person. And so the scores there was the award ceremony and we didn't know what what happened but the there was someone else who was um you know what what what did the what was everyone's scores over time look like and there was you know say 64 I think there were 32 actually people playing so it was like around uh 32 kind of tournament and 30 people over time you know their scores would were all very negative and going down and then there was one person whose score was like like pretty much straight up and then there was another that was like pretty good but but not like with a crazy slope. And um so do do you want to guess which which one we were? So so we were we were the lower slope and then there was this other guy that was had this crazy slope was just like completely crushing all the other players. And then this happened for the round of 16, the round of eight, the round of four.

Dan Roberts: And then in the round of two, it's heads up, us versus this guy who's like over the course of this tournament won way more than us uh overall like taking more money from from everyone else. And then we crushed him because why? Because he was exploiting the weaknesses of of of everybody else, right? It it's had some theory of mind to try to figure out oh this this guy you know does this when he bluffs and and so it was very I assume it was very good at at like taking um you know exploiting everyone else but we were just playing the best possible thing that you could that you can do given you know so the the criteria was not maximize your your amount that you get from anyone else it was don't lose like play you know and and uh and you know so it's the best response to anyone's strategy and so at the end we had to win assuming we did it right and someone else playing the same strategy would would tie.

Matt Turk: Okay. Fascinating. So just tying this back to the beginning of the conversation about air do problem and and solving uh unsolved math problems. So presumably the instinct would be that you need a lot of exploration not exploitation. So how does how does that work in the context of novel scientific discovery?

Dan Roberts: I think math research or scientific research in general has a lot of versions of both explore and and and exploit. um to give to give the recent example, the OpenAI unit distance uh proof I think is is is is very much in the explorer setting where the the model was happy to be contrarian and and try to disprove this thing that everyone believed and it was it was just looking for it has this huge repository of understanding all of human math. And so it was it was spending a very long amount of time. I forget how how many hours but I think we published a rewritten version this chain of thought but like hours and hours trying different things. So it's clearly in the in the domain of exploration.

Dan Roberts: Um a lot of times though you can ask these models to uh compute something that they understand very well and then that that has a different structure and might look a lot like exploit. There's a a paper that came out recently after the open AI result where the uh an unrelated problem. It has something to do with uh if you have a set and you try to add it to a cell add the set to itself or you try to multiply the set with itself. So like take the elements and add them all together or take the element individualize or multiply them together and how many unique sums or products you get. There's some conjecture around that. And this this one was also disproved and and that was done by by humans and the core idea was um it's like a totally different problem but there there was inspiration from the from the unit distance one the idea that you you can sort of generalize uh from from the as pick a pick a certain find pick a certain type of of numbers that had a certain property that that the open AAI model figured out and that they realized that like this applies in this setting. So that's very much an exploit thing.

Dan Roberts: But um and so so I think the process clearly is like this the actual discovery process. Uh I think normally when you talk about explore exploits maybe we're talking about when training reinforcement learning models how should we train them but I think there's this interesting point that in the scientific discovery process there's really this interplay between like exploring exploration and then exploitation in order to totally push the field forward.

Dan Roberts: switching to RL in modern LLM systems. So there used to be a saying which I think comes from Yanukan that RL was the cherry on top of the cake, but I think you have argued that things have switched now and that uh RL is the main part the the cake. Do you want to just walk us through what you were thinking?

Dan Roberts: Yeah, I said that about a year and a half ago. Had to give a talk that was public and I couldn't say much. So, I decided to invert this meme with the with this cake and the cherry. RL is really exciting. That's why what I'm here talking about. And I think that when you have a lot of compute, you want to turn that compute into intelligence in a way that's that's useful. And RL is one way of doing it. and we just started doing it then and we're going to do a lot more of it now.

Matt Turk: Why did RL start working? Well, it's not an entirely new concept is uh been tried for many years now. Uh what is different now?

Dan Roberts: But there's an interesting thing to to point out here though which is that uh there there's this question of how do you get models to think in test time and and reason and there was a reasoning effort at OpenAI that was quite early and spent some time and and came up with some some algorithms. I think maybe the simple thing to say is that if you have a powerful enough pre-trained model then it can start to do well at RL. it can start to like think um at at use test time compute to to for instance solve solve math problems that it that it wouldn't otherwise be able to do.

Matt Turk: This um a viral analysis uh from earlier this year February I think that claims that um RL produces less than one bit of useful information per 10,000 tokens and then Carpathy called it sucking supervisions through a straw. What is your take on on this and the overall efficiency uh of of a

Dan Roberts: if you look at like the deepsek algorithm which is a public thing that that we can talk about then the you train on on um sequences that that are correct. So whether it's correct or not is maybe one bit of information. So I I think there's you know you can see where that logic comes from. I think the question is like is this doing a kind of thing that you can't otherwise do? Like maybe you would want to give more supervision, but how how are you going to do that? I think I think it's very clear that that this method these methods have lead led to a bunch of of breakthroughs um in terms of like the explosion of what the models can do and and you know both in in coding and in science. I think broadly it's about getting models to think in in test time to use test time compute and and and and do reasoning. And there's clearly a lot of the pieces of what the RL process is that's essential to make that work.

Matt Turk: What's your overall feeling in terms of um how far we can go uh with that current sort of systems model where we have pre-training and then we have um RL on on on top somewhat famously last year there was a conversation with Rich Sutton on the Dwarkish podcast where his uh claim in my best attempt to paraphrase it was that LMS were not really intelligence and uh therefore for uh RL was the only way to do it and R pure RL not LM plus RLS. What is your take on this? I mean obviously you're in on an RL team at a company that does uh you know both pre-training and RL combined. So what's your what's your take?

Dan Roberts: But then the linguist was was like blah blah blah Victenstein, you know, the everything goes through language. That's the method of communication. That's, you know, that's the way words mean things are are um the central thing. And and when we want to talk about the laws of physics, we have to use language. And I sort of feel like he and Victor like that that was correct, right? That's what's what or at least the path through AI suggests that that is a correct path. I'm I'm conceding now to Kyle and u if he's if he's listening, he's now a linguistics professor. This whole idea of reinforcement learning that kicked off the previous decade's interested in AI. the the sort of uh grounding I think that was needed was was to to make things really work is through is through language because everything goes through language right all the internet right it's like it it incorporates the grounding of the real real world all of our scientific knowledge all of our mathematical knowledge all all of the human like you know the sum total basically of of human work is represented on the internet in language and then so having the model have a prior of language and being being able to like think in think in language and then train on top of that. That seems like clearly the right the right thing to do and and like you know seems also well well grounded in a way that even before all this somebody might have argued would would make sense. It's like an amazing prior to have um to to start with for for an intelligence because it's like very much based on us and our society.

Dan Roberts: I have other disagreements with with uh Rich Sutton but if you want want to poke at that.

Matt Turk: Yes. So just give us one one or two quick ones.

Dan Roberts: I have a somewhat contrarian take that that um with the better lesson that it's not that scale is all you need. You need to also have have good ideas to to guide the scaling. So there's there's like a deeper interplay than just scale things up. For for instance, if you were just trying to scale pre-training, you wouldn't get anywhere near as far as also trying to scale RL on top of pre-training, which is what we do now. And our models are much more powerful for like that very good idea and investing in that good idea.

Matt Turk: Since you mentioned test time compute, I think there's um something that that still puzzles people which is the the whole chain of thought thing which is so magical from a user perspective whatever you can see what actually happens during test time compute uh that creates uh those um artifacts. What does a model actually do?

Dan Roberts: I I think it does what what you see it do. um we lightly rewrite it or summarize it. Um but it it just it just produces tokens and those tokens are like a running thought process just like you might have or or maybe it's more akin to if you're solving a math problem the the scratch pad the collection of notes that that that you have but it it just keeps generating. The cool thing about generating is that you know it it it's a a forward pass the model. So we're using a bunch of computation. So we're we're you know it's a way of leveraging a lot more computation on a problem than than you would before.

Dan Roberts: So my colleague Nome Brown likes to talk about the remon hypothesis a lot and you know wouldn't you want to have a model that runs for years that that that can um resolve resolve that prove that if you present it and you wanted to produce an answer then it only has the number of flops in a in a single forward pass to produce one one token if it's forced to answer right away. But if it gets to answer after think you know after a long time it can it can re reuse its weights you know produce um a final answer that is a function of a m much much larger amount of computation and the like the natural way it thinks is in language it's a language model and so that's sort of this key insight that that you can um cause it to do better just by producing a thought process in in in token space in in language.

Dan Roberts: and this was known before RL the idea that if you asked the model, if you gave a model examples of of thinking things out, it would do this before it produced a final answer. Or if you just told it that, then it would then it would do this sort of thing, right? Going back to this SFT versus supervised learning versus reinforcement learning analogy that I gave earlier, like there's a lot of examples on the internet of people thinking for a long time. And so like it's not completely useless. It can channel that a bit, but RL really like brings that that out.

Matt Turk: What happens during testm compute is is RL related or created because that that's effectively what you described earlier when you were defining RL the model goes in one direction decides maybe that's not a fruitful one backtracks tries something else is that correct or not

Matt Turk: So one of the key questions in the field is whether you can expand and generalize the success uh that LLM systems have had in particularly in coding and now math uh but like domains where you can sort of verify uh whether uh what the model comes up with is is correct or not. What is your view on on that and and perhaps start by explaining what a ver verifiable reward is?

Matt Turk: So do you think there is a path for RL to be truly effective at domains without referral rewards? So uh you know consulting, banking, uh legal I mean clearly there's tremendous progress in those domains but like what what is happening?

Dan Roberts: I definitely think OpenAI will have amazing products that will um be relevant in those domains and some amount of RL will play a role in there.

Matt Turk: Does RL generalize uh meaning that as you train it against more and more domains it becomes disproportionately good at learning the next domain? I mean we want to make a model that is generally intelligent and push that intelligence as far as possible and to do that we want to make everything part of the distribution and then we also want to make it robust in cases where it encounters things that it was not in the distribution and if RL is part of that process then you know like but like I think there's a vague sense as I was trying to say earlier is that there's a lot of things that are very fuzzy but the you know clearly the question of generalization in AI is is is a important central one and there's a bunch of examples I think that support that the processes can do this.

Matt Turk: So going back to your physics roots, a lot of what we just described about this interplay between pre-training and RL and uh you know all the various bits that we described, those are clearly pretty complex systems. Uh and uh you uh were trained in a discipline that is all about studying complex systems. What can physics uh teach us about how to understand uh those uh those uh AI systems that we're currently building?

Dan Roberts: I think there's a lot of lot of angles to answering that question. I think the maybe most interesting one or the most relevant one to how we work currently and maybe this is a contrarian take is that the way to think about scaling and and say scaling laws is not small to big but big to small. So and and I'll get to why physics really matters for this in a second. when when you have you know you have the existence of some really big AI system and some weird things happen and they didn't happen at the small scale and so we say like oh this whatever emerged at scale right there sometimes people use the word grocking or there's something discontinuous about the scaling sequence and so it see or the scaling law is broken these are things that people might say but I think I reject that entirely I think you it means that you didn't understand something about what you were scaling up.

Dan Roberts: Go back and make smaller and simpler models or or simpler toy examples such that the um the whole thing is smooth. And if you can do that, if you can figure out what to put into the small thing, then then you understand the thing um and and then and then you can you can move forward. This is exactly what we do in theoretical physics. There's the standard model which is you know I I have it a textbook behind me it the description of all the forces except gravity would take even in compact notation like the entire entire page it's it's like completely gross there's you know a lot of different particles why you know I who knows some of them there's reasons for but they're doing all sorts of different things different things cancel whatever or you know this just happens to be the universe that we live in but you don't need all of that to to like study pieces of it to study electromagnetism, you forget about everything else. Or if you want to study the Higs phenomena, you know, which gives mass to some particles, you you can study a simplified version of that.

Dan Roberts: Um, and and and so what we do in I think one of the key moves in at least in my training in physics is to take really complicated systems. This often gets talked about as physicists just study spherical cows. And and I think that that that like kind of misses the the point like you you study if if the spherical cow is sufficient to describe the thing that you care about then you did a good job and if not you did a bad job. You don't try to retreat to a setting that's simple enough where you can make pro where you can like calculate something. You try to retreat to the setting that's simple enough that contains the thing that you care about and then you you have no idea whether you can make progress there or not. But once you did you sort of understand what the problem is and you and and that's that's like uh a lot of the work in physics and I and the same thing is true in AI. You have these crazy huge systems that have all sorts of interesting phenomena and and you know if you think about it the right way they don't grock there's just this nice continuity.

Matt Turk: Do you think there could be an equivalent in AI to thermodynamics meaning uh you know a compact theory that uh predicts behavior without tracking every individual bit.

Matt Turk: so at that conference a year ago you you jokingly uh predicted 9 years to Einstein level AI where do you think all jokes aside we are on that on that spectrum of uh just um AI uh uh creating scientific discovery. I mean that's where we started the conversation and curious about where this is going.

Dan Roberts: The joke maybe it's helpful to deconstruct a joke as as as it always is. But the the joke was that taking the doubling time for uh the amount of work a system can do autonomously and figuring out how long it would take us to get to a system that can think eight years on its own because Einstein spent eight years discovering general relativity. and I projected that out and it was like nine years from last year. So, um that that like something I I hate making predictions, but I'm pretty sure something will break before that. I I mean, in in general, we're not just going to like set up a system and let it think autonomously for 8 years, if anything, because like the systems 8 years after will be so much more powerful that it probably doesn't make sense to let a system think for a certain amount. you know, there's there's amount of time it takes for the system to improve and then there's the amount of time in it thinking and like probably when those cross like all these scaling walls are going to to break in in in certain ways.

Dan Roberts: I do think that the kind of thing that that I was trying to talk about about like how we as physicists approach problems that the structure and flavor of that is maybe different than here's like a very well- definfined thing and go and do a calculation which is like what these Erdos problems are. I think probably we'll need to have some ideas to bridge from from one to the other. I don't think it's it's not obvious whether there has to be a discontinuous thing or or or a smoothing thing, but you know there's part of the scientific process I think that the models haven't been imbued with yet and I'm sure people are thinking about how to how to how to do that. You know like what trying to get to what is the right question as opposed to here's a well- definfined thing and and and go calculate and and some of that involves research taste that's not an easy easily verifiable thing.

Matt Turk: Is that what would convince you that AI is doing genuine original science?

Dan Roberts: No, I'm convinced and I think we're going to like I like like this is clearly I think the the unit distance problem is a is a is a great example and and like the also just being able to take a position that is contrary and think for an extremely long amount of time explore lots of different options and bring to bear the full weight of disparate fields like where something you know it's very unlikely to find a human that has the exact set of skills to solve some of these problems right that's a huge that's a huge thing

Matt Turk: how far do Do you think we are from AI research uh actually automating itself uh not not just AI researchers using AI but like AI autonomously building AI?

Dan Roberts: Yeah, I think it's again one of these smooth things where where like it's already doing pieces of it it now. It'll do more in the future. And there's I know there's strong versions of this that that that that people like to think about, but you know, I I'm not sure that we'll see like a a really sharp phase transition versus just more and more pieces of you know, right now a lot of coding that would take people weeks can be done very efficiently with models. So like some of these math discovery problems, there's also versions of this where for engineering the the model the models are playing a more central role. And so I think that will just there will just be more of that. I think that there's a kind of scientific thinking that that humans still seem to be very useful for doing and you know I don't want to make specific predictions about when or how.

Dan Roberts: And so but in the you know in the I can make a very long distance prediction you know for the next 6 months like I think we'll see more of of these sorts of math and science breakthroughs and and obviously we'll turn this sort of thing on on AI itself and the models will get a lot more powerful and that'll that'll be fun. You could think about you know that you could do science of AI and have it feel like doing doing physics and and and that's true. Another really exciting thing is that like I entered physics thinking that I would you know when you you first start learning a field and and maybe you want to commit to it at least the perspective I had is that oh by the time I get to the end I'll know all the answers right all the all the fundamental questions obviously like this is a journey and at the end of the journey it'll resolve and then I don't know maybe it was in grad school or maybe when I switched to AI I realized oh like some of these questions will stay open maybe forever maybe I'll never get to learn the answers you know watching older colleagues as well start to retire and and realize that they might not get to learn the answers. But I I feel really excited that that you know we will get to really answer a lot of fundamental questions in in the fields of science that that we care about um with with the aid or maybe the the models being the driving force and so that yeah that that's just really thrilling.

Matt Turk: Well, that feels like a wonderful place to leave it. Dan, you gave us plenty to ponder. really appreciate you spending time with us today. Thank you.

Dan Roberts: Thanks for inviting me. It was a pleasure.

Matt Turk: Hi, it's Matt Turk again. Thanks for listening to this episode of the Mad Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already, or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks, and see you at the next episode.