Unknown Speaker: there's a future where like the distinction between small and uh large models like disappears to some degree um and with long context there's also a degree to which fine tuning might disappear to be honest um

Unknown Speaker: like the these two things that are very important today like today's landscape of models we have like whole different tiers of model sizes and we have fine sh models of different things you can imagine a future where you just actually have a dynamic bundle of compute and uh like infinite context um and the that specializes your model to to different things one of the bottleneck for AI progress that many people identify is the inability of these models to perform tasks on Long Horizons which means engaging with the task for many hours or even many weeks or months where like if I have I don't know an assistant or an employe or something they can just do a thing I tell them for a while um and AI agents haven't taken off for this reason from what I understand so how linked are long context windows and the ability to perform well on them and the ability to do these kinds of long Horizon tasks that require you to engage with uh an assignment for many hours or are these unrelated Concepts

Unknown Speaker: I mean I would actually take issue with the that being the reason that agents haven't taken off um where I think that's more about like nines of reliability and the model actually successfully doing things and if you just can't chain tasks successively with high enough probability then you won't get something that looks like an agent that's why something like an agent might follow more of a step function like gbd4 class models G Ultra class models they're not enough um but maybe like the next incr model scale means that you get that extra nine even even though like the loss isn't going down that dramatically that like small amount of extra ability gives you the extra and like yeah obviously you need some amount of context to fit long Horizon tasks but I don't think that's been the limiting factor up till now

Unknown Speaker: do you expect that it will be multiple copies of models talking to each other or will it be just uh a adap a computer solve then the thing just like runs bigger uh like more compute when it needs to do a kind of thing that a whole firm needs to do and I asked this because there's two things that make me wonder about like whether agents is the right way to think about what will happen in the future one is with longer context these models are able to ingest and consider the information that no human can and therefore we need like one engineer who's thinking about the front end code and one engineer is thinking about the back end code where this thing can just ingest the whole thing the sort of like keken problem of specialization uh goes away second these models are just like very general um of you're like not using different types of gp4 to do different kinds of things you're using the exact same model right so I wonder if what that implies is in the future like an AI firm is just like a model instead of bunch of AI agents hooked together

Unknown Speaker: that's a great question um I think especially in the near term uh it will look much more like agents look together and I say that like purely because as humans we're going to want to have these like isolated reliable and uh like like like components that we can trust so if you're claiming is that the AI agents haven't taken off because of reliability rather than long Horizon task performance isn't the lack of reliability when a task is changed on top of another task on top of another task isn't that exactly the difficulty with long Horizon tasks is that like you have to do 10 things in a row or 100 things in a row and diminishing the reliability of any one of them uh or yeah the probability goes down from 99.99 to 99 .9 then like the whole thing gets multiplied together and the whole thing becomes much less likely to happen

Unknown Speaker: one of the things that will be really important to do over the next however long is understand better what does success rate over long Horizon task look like um and I think that's even important to understand what the economic impact of these models might be and like actually properly judge increasing capabilities right by like cutting down the tasks that we do and the inputs and outputs involved into minutes or hours or or days and seeing how good it is successively training and completing Tas those different resolutions of time because then that tells you like how automatable a job family or task family is um in a way that like MMO you schooles don't

Unknown Speaker: I mean it was less than a year ago that we introduced 100K context windows and I think everyone was pretty surprised by that so yeah everyone would just kind of had this sound bite of quadratic attention costs and we can't have long context windows and uh here we are so yeah like the benchmarks are being actively made

Unknown Speaker: one thing you can imagine is you have an AI firm or something and the whole thing is like end to end trained on the signal of like did I make profits or like if if that's like too ambiguous if it's if it's an Architecture Firm and they're making blueprints did did my client like the blueprints and in the middle you can imagine agents who are sales people and agents who are like doing the designing agents who like do the editing whatever um uh would that kind of signal work on an end toin system like that

Unknown Speaker: yeah in the limit yes that's the dream of reinforcement L right it's like all you need to do is provide this extremely Spar signal and then over enough it already you sort create the information that allows you to learn from that signal um but I don't expect that to be the thing that works first I think this is going to require an incredible amount of care um and like diligence on the behalf of humans surrounding these uh machines um and making sure they do exactly the right thing and exactly what you want and giving them right signals to improve in the ways that you want yeah

Unknown Speaker: you can't train on the RL reward unless the model generates some reward yeah

Unknown Speaker: yeah yeah exactly you're in like you're in this like RL world where like if it never the client never likes what you produce then like you don't get any reward at all and like it's kind of bad but in the future these models will be good enough to get the reward some of the time right this is the nines of reliability that was talking about yeah