Joel Becker: Here's the very simple argument. If you look at the sub notion of compute over time—this could be like R&D spending on compute, this could be experimental compute, it could be training compute, whatever some particular lab is using—it goes like this, no surprise. If you have another chart of like log time horizon, let's say this METR measure from the figure that many of you would have seen on Twitter over time, it looks like these things were causally proportional in the sense that if compute growth were to half, then time horizon growth were to half. So for the sake of argument, let's say that starting from '28 or so, the compute curve begins to bend like that where this would be no growth and this would be the original growth, something like half, then if they were causally related and in particular they were causally proportional to one another, then you'd expect this to go like that and then the delay implied in AI capabilities is potentially enormous.

Now like why lots of people have stipulated that there might be some slowdown in compute growth, I'm not an expert in those forecasts but I think the prior reasons do seem like somewhat strong to me. One is like physical constraints that we might hit power constraints as you mentioned or there are various other ones that EPO have a report on that they consider, all of which seem to not bite through 2030 but potentially could bite sometime after 2030. I think the more likely one is just like dollars is a constraint, like large tech companies can only spend so much at a certain point, like large nation states can only spend so much. The additional point that this paper is trying to make is that under a very contestable but standard assumption from economics, you should in fact expect these two to be causally proportional, at least for the period that software-only singularity is not possible. I think this is maybe a reasonable model and does imply some sorting of AI capabilities in the near future.

Interviewer: That also tells us we don't have a technological advance that dramatically improves capabilities relative to like an unpredictable technological advance, right?

Joel Becker: Yeah. I mean, all predictions assume no unpredictable... in general in AI, kind of straight lines on log-linear plots have been a very highly underrated forecasting tool. They've done extremely well over now many orders of magnitude. I think it's reasonable to have the default expectation that the log-linear lines continue through approximately the same number of orders of magnitude except maybe if there's some significant break in the inputs. Yeah, of course on the upside there could be something quite dramatic. Software singularity is the first thing that comes to my mind, but another Transformer-style moment seems like another candidate naturally.

Interviewer: One thing I thought and you brought up a little bit in the paper which is whether or not familiarity is a confounding factor. There was an interesting presentation from Meta at the developer activity engineering summit this year. They have probably the best infrastructure for quantitative measurement of developer experience in the world of any company. They're able to tell you basically how long it actually takes to make a PR, how much actual human time effort it took. What they saw was a J-curve when they gave people agents, and that J-curve was like three months or six months. So one of the things I also wonder is if there's a cut-off of how much familiarity the person has—have they been using this as their full-time daily driver for a period of months—and if there's like a cut-off that occurs once a certain level of familiarity occurs?

Joel Becker: I'm totally on board with J-curve explanations being a real thing. Developers tend to be slower the first time that they're experimenting with tools. You're doing this so that you have some investment benefits later on. One thing to say is we see this over and over in software engineering studies that the one question you can't ask people in a survey is "how long did a task take"—ask anybody the amount of time that something takes, they are almost always wrong. But you can ask people "how much more productive did you feel" and they will give you an accurate response that correlates to quantitative feedback.

Joel Becker: We watched so many hours of screen recordings of these developers working and I just do not see... I think they're prompting very reasonably. I'm not seeing these like advanced workflows that they're not accessing. My experience is not that far off from this is that there are times when I am dramatically slowed down and there's times when I am accelerated. Although as my familiarity with the tool increases, I definitely improve a lot because I learn over time what I can tell it to do and what I can't tell it to do.

Interviewer: That is a really interesting point because actually some of the repos that I was helped the most with were ones that I was completely unfamiliar with and which had no decent documentation. In that case, Claude was a huge help. Joel Becker: Yeah. Legacy code bases don't exist because they work well. It's because they make money.

Interviewer: If I was going to pick the areas where I would expect to be more successful, but where I think it is being less successful, I would pick probably data science. At LinkedIn, there are 5,000 tables with the name "impressions" in the table. If an analyst wants to understand how many impressions happened on a page, where the hell do they go? AI can't figure that out. I believe that the state of underlying data is so bad that the actual data scientist is going to get way less value out of the AI than software engineers.

Joel Becker: That's very curious. One view that some more bearish people have looking at the future of AI is there's so much tacit knowledge embedded inside of companies that you're not going to pick up from training data. Interviewer: Another one I'm curious about is lawyers. Discovery in law is a huge problem. You could actually have significant impact on a society if you could make discovery cheap and instantaneous and reliable.

Joel Becker: Why are robotics capabilities lagging LLM capabilities so much? Is it to do with training data or maybe it's to do with hardware constraints? If we put super intelligence inside of hardware parts that existed today, could it build chip production facilities? Interviewer: I have a friend who started working on manufacturing and he found out how hard the manufacturing world is and how slow the iteration process is. It's an order of magnitude worse. The rate of improvement is actually glacial compared to software.

Joel Becker: Is that really your view, centuries? Interviewer: My guess is it's going to take a lot longer than we think, at least to be able to do like real mass production. Joel Becker: Maybe we're in a sort of GPT-2 moment. There's a lot of opportunity for AI in chip manufacturing, like calculating the mask for the laser or improving yield by detecting parameters that are out of whack.