Benedict Evans: If you're Sam Alman, you've got a commodity technology. You've got you're competing with people who have giant legacy cash flows. You don't have your own infrastructure, don't really have any differentiation, but you've got massive mind share. So, what do you do? You try and swap that for hard assets and you try and talk your way into a self-fulfilling prophecy.

Benedict Evans: And then you've got this middle case where you could almost kind of call it like improvised software. And now AI coding means coding is way cheaper and easier and also means that there's a whole bunch of stuff that you couldn't do in software that now you can. And so there will be way more software.

Matt Turk: Hi, I'm Matt from First Mark. Welcome to the Matt podcast. Today my guest is Benedict Evans. Ben is one of the most thoughtful and influential tech analysts in the world. He's also one of my favorite guests making his third appearance on the show today. In this conversation, we cover the great AI unbundling, whether software is really dead, why OpenAI might be the next Netscape, and what it all means for AI builders, investors, and executives. Please enjoy this fantastic conversation with Ben Evans. Benig, welcome or I should say welcome back. Thanks for being back on the Matt podcast.

Benedict Evans: Thank you.

Matt Turk: So I thought a fun place to start would be what's going on with OpenAI. So, as we recording this just last night, there was a Wall Street Journal article that said that the company is planning a major refocus on coding and business users with their CEO of application Fiji Simo reportedly saying that the company should stop side quests. You wrote recently about OpenAI. How did we get here and what are the fundamental challenges that OpenAI faces?

Benedict Evans: I suppose there's two way you ways we could talk about this. One of them is to talk about all the stuff that OpenAI has been trying to do. I think the other is to talk about the problem. And the problem is that as as far as we can see, there is no winner takes all effect or network effect in a foundation model. So there's nothing that you can do that means other people can't make a model as good as yours, which is what we're used to from the software industry.

Benedict Evans: That's how Windows and Google and Facebook and Instagram and Tik Tok and iOS. What we're kind of used to in tech is that software by definition has no capital or very little capital but it has network effects and so it tends to produce monopolies or near monopolies and that produces high margins and so that what happened with Windows and Mac OS and Google and so iOS not Mac OS and Google and so on. Well, the question with LLMs is like they're very expensive and very hard and so you can kind of fail to get onto the ladder or fall off the ladder for like Microsoft on the one hand and Meta for for the meantime on the other hand. But there's nothing that there's no lever that you can decide you're going to pull whereby you'll just pull ahead of everybody else and they won't be able to catch up which is like Google versus Bing. Doesn't matter how much money and how hard Microsoft works, Bing will never catch up with Google. So that means if we've got like pick a number between three and six or maybe more organizations that can make a frontier model and they keep leaprogging each other every couple of weeks or every couple of months. So that's one problem.

Matt Turk: Just to build on that problem, you mentioned software. Is it such a bad thing? If you think of not Windows, but if you think of the oligopoly of AWS, Azure and GCP, those are largely undifferiated businesses that in because of the market size seem to be doing quite well.

Benedict Evans: First of all, I push back slightly on that comparison and then then answer the question. And the first is if you actually look at the market shares, Google cloud, Azure and and AWS are actually in quite different businesses. AWS is mostly infrastructure. Microsoft is mostly services. Google is mostly scrambling to catch up in in a very distant certain place despite having basically invented cloud. So they're kind of not all the same, I think. But the the challenge I suppose is is where do we get to equilibrium on the foundation model itself?

Benedict Evans: Is it that these things will be able to create ecosystems above themselves? So it will be like iOS and Android and so they will create value they will create value capture they'll build tools and all sorts of stuff on top of the raw foundation model and so you'll have to choose right all our company is going to standardize on chat GPT which isn't what happened in cloud you as a business if you go and buy a SAS app you don't like think well we're going to log into it with our AWS account certainly as a consumer like I don't can't remember which cloud snap uses and I don't care and if I you I install Uber like which cloud does Uber use? Who cares? So it may be that the models are able to kind of build something that looks more kind of like Windows which is what Sam Waltman talked about a lot at the end of last year or it may be that they're basically commodity infrastructure.

Benedict Evans: and they're sold at marginal cost and maybe you barely make a return on the investment or maybe you don't make a return on the investment itself and what you're doing is making a return on everything you built that you do with this on top of it. And that's where the the the these companies do get different because if you are Meta or Google, you've got this whole other highly profitable business which now needs to have LLMs inside it powering all sorts of capabilities and features. And you probably want them to be your LLMs rather than somebody else's. But you not may not necessarily need to make money from the LLM by itself anymore than Meta. I mean, obviously Meta has a cloud, but Meta doesn't even try and doesn't sell the cloud. they just use it power their own stuff.

Benedict Evans: well the problem at the moment is that the raw chatbot by itself isn't a great product and most people struggle to work out what to do with it you've got um a small number of people mostly in tech and also outside tech who are very self-optimizing and have certain kinds of jobs that map very well to the kinds of stuff that LLMs are very good at doing. If you look at the usage data, something like 10% of the population is using these things every day, but another 50% are using it every week or every month. So most people who have a chat GPT account can't think of anything to do with it today.

Benedict Evans: And so you can ask, well, is that because the models have to get better? Or is it because habits have to change? Or at the extreme, could you say, well, people only use Google once a month, once a week, too, which doesn't seem like a good answer. Or do you have to build a whole bunch of stuff on top, which is back to this commodity infrastructure point? And so it seems like what a lot of what the question now is like, oh, are we all just going to use chat GBT or anthropic or or Claude as chat JBT or is that going to be like an invisible API call buried underneath? that's used by some other thing and it's the other thing that makes all the money.

Benedict Evans: The really kind of interesting comparison here would be TSMC because what happened in chips was that there's no network effect but with each generation it got more difficult and more expensive and now there's basically one company at the frontier and two or three that are a generation behind and then five or 10 that are a generation behind that and TSMC makes lots of money but like they don't capture all the value of the tech industry like you know as a consumer you don't know that TSMC makes a chip in your iPhone and they don't get like a cut of every app store sale. So that's sort of the puzzle. This great strategic problem for Open AI is you basically got commodity technology. You don't have a network effect or when it takes all effect. You've got 900 million weekly active users, but most of them are not using it every day and can't think of anything to do with it. And only 5% of them are paying for it. So the usage is like a mile wide but an inch deep. And so you've kind of got to swap the mind share and the momentum that you have for something more durable.

Matt Turk: Yeah, you just said u last year when you and I chatted I think we were having this conversation in those terms that like people were not sure what to do. We went into whether JPD needed a new guey. What's fascinating is that it's been an extraordinary year in model development with you know reasoning and RL and and all the things

Benedict Evans: and the net is that a better model doesn't solve the problem. This is something I wrote like a year ago I think where I said what do you mean when you when you it was something like what is a better model and which sounds like a crazy thing to say but I always remember um like years ago when iPhones was a hot new thing um one of the New York late night TV shows they would go onto the street and they show people last year's phone and say it's a new phone what do you think and they go wow it's amazing cuz like you couldn't really tell actually

Benedict Evans: and you know if you're doing like very specific very hardcore stuff and you're pushing these models to the limit then you'll be able to Oh, it didn't do that and now it can do that. With most stuff, the models are jagged in lots of different ways. You don't really know whether it will be able to do that or not or how well it will be able to do that. And most of the stuff that you try, it can sort of do to varying degrees. And now it can sort of do it slightly better or maybe slightly less sort of. But if you've got a bunch of use cases where you need the right answer as opposed to sort of the right answer, then saying that the model is better doesn't mean anything. I mean literally is literally meaningless because what you're telling me is I asked the model to compile 50 things and last month it would get 10 of them wrong. So I'd have to check all 50 and now it will get eight of them wrong or 12 but probably eight. So again I'll have to check all 50. So actually nothing's changed. And the point that things change is not when it goes from 90% right to 91% right to 95% right. The point it's either right or not right.

Matt Turk: At some point there was a hope that memory would be adding some level of defensibility and mode to all those uh chatbot interfaces including very much chat GPT. That doesn't seem to have happened. Why is that?

Benedict Evans: Well, so two or three things. Firstly, I think very obviously it only works if you're using it a lot already. And so if you haven't worked out a bunch of stuff to do with this, and most people have not, then you don't see the memory. OpenAI did this this data at the end marketing at the end of last year where they gave everyone a cute little graphic of how many messages they done... And it turns out that if you did a thousand prompts last year, you're in the top 20%. So basically 80% of people hit return less than a thousand times last year... And so there's no memory there. Me then the sort of a subsidiary point would be okay is that a network effect or is that stickiness? It's probably more stickiness.

Benedict Evans: One thing that occurred to me looking at chat GBT as a product is it's kind of like trying to trying to differentiate the web the chatbot itself is kind of like trying to differentiate a web browser in that you've got an input box and output box. And how can you make them different if the whole point is that you can type in anything and get anything out? ... Is it even possible in principle to make the UI of a of a chatbot different? Because isn't the whole point that it's completely universal and there's no UI?

Matt Turk: yeah you had a really interesting point somewhere about how building product in a foundation model company was fundamentally different from any other company.

Benedict Evans: Oh yeah. Well, this is so so Kevin Wheel and and Mike Quigger gave this made both made this point... And then Fiji Simo made it... she said... the way it works is you you turn on your computer and you you look at your phone in the morning, you've got an email from the research group that says, hey, guess what? We've got this cool thing. And then your job is to go and do something with it. ... You don't control the product strategy, which is of course how science works... Simi... comparing it with Steve Jobs famously saying you can't start with the technology and work forward to the product. You've got to start with the user experience and work backwards to the technology. ... You're a strategy taker not a strategy setter.

Matt Turk: Yeah. And to the point that you're just making and and also anthropic presumably you're not in the camp of believing that people are going to be building their own software with claude code.

Benedict Evans: It's funny this is like um it's almost like a straw man except there are like presumably real rational sentient people who actually say this stuff. I don't think it's particularly interesting to explain why people aren't going to code their own stripes. I think what's more interesting is to kind of do think about a broader taxonomy of software which is that you have like the big eye and enterprise ERPs and and what they call systems of record. So like you need to store the same data in the same way for 50,000 people and a billion transactions a year. ... Then you have but then you have tasks and workflows where that single system like call it SAP is too inflexible and so you break it out unbundle it into something dedicated and vertical.

Benedict Evans: and then you've got this middle case of using Google Sheets and email or using Excel... where you could almost kind of call it like improvised software where it's not a process that's being done over and over again in the same way... And now on the one side, AI coding means coding is way cheaper and easier and also means that there's a whole bunch of stuff that you couldn't do in software that now you can. And so there will be way more software.

Matt Turk: so what do you think happens to software than than SAS?

Benedict Evans: way more software. ... Who understands how software works and who has really thought carefully and understood the problem... Frame.io was not made by some guy or some woman working at a video production suite... because that's not how they think. They're not software people. They're also not product people. ... There will be way more software both stuff that doesn't need AI except to create it and stuff that needs to use AI to do the new thing. Some incumbent software will go away... It is just like a one-off improvised thing where you'll get the model to do it.

Benedict Evans: people last year suddenly everyone discovered looked up the Jeffness paradox in Wikipedia... this is just pricey elasticity... if you make it cheaper and easier to do something you might do the same thing for less money or you might do more for the same amount of money or you might do do more with more money if you have a completely different ROI which is exactly what you see in like financial services is spreadsheets did not result in a collapse in the number of people working in finance say quite the opposite.

Benedict Evans: the market end reason software is eating the world thing of Uber and Airbnb... Uber rides per day is like triple double or triple what yellow cabs were... Airbnb was mostly additive. And you dig into that and the answer is why... Benedict... always says the answer is it depends. ... travel is business... she's not going to go and stay in an Airbnb. Like, absolutely zero negative possibility that she's going to go and stay in an Airbnb.

Benedict Evans: So, this is the problem I have with people kind of trying to score professions, AI exposure... directionally right in the same way that an analysis you did in 1997 about the internet would have been... But you would not have got Uber from that analysis. You would have said, well, taxi drivers. Well, how would the internet change that? Obviously won't touch that one at all. You would have said newspapers maybe. ... what really happened was you unbundled physical assets from the underlying product. And if your defensibility was based on owning a physical asset... then your whole business model has just exploded.

Matt Turk: and you talk to a lot of big corporations... What's your sense of the overall sentiment? Are people more bewildered than they used to be?

Benedict Evans: So, I certainly wouldn't say bewildered. I mean, at least not any more bewildered than anyone in tech. ... Everyone deployed CPilot and went, "Oh, okay. That wasn't very successful." Every It's kind of like giving everybody the internet in 1997 and saying, "There you are. Be more productive. Doesn't didn't really work." Everyone now has a bunch of pilots.

Benedict Evans: Paradoxically, the answer for a builder is actually super easy... Frame.io... In principle, you could have had that idea at any time in the previous decade. ... Most SAS companies are database rappers where somebody realized that here is this problem... And the hard part of that was not writing a bunch of SQL queries... It's all the other stuff around like what what should the code be doing and how would we tell people... and I don't think that has changed at all.

Benedict Evans: I suspect that it's the third category that produce the best outcomes. ... Yes, you can pick up AI and go around looking hitting everything with it. You know, it's a hammer and everything's a nail. Or you can start from the other end and think, you know, now finally I can solve that problem.

Matt Turk: All right, Ben, but we could quite literally go on for hours. This is so fascinating. ... Thank you so much for being back on the Mad Pod and um look forward to the next one.