[Matt]: There was a moment in time in Silicon Valley a few years ago. If you had asked most people, they would have said that Neoclouds were going to be a commodity in particular because GPU compute was going to get commoditized. And if you fast forward to today, it seems to be exactly the opposite. Both Lambda but several of your competitors seems to be absolutely ripping. So what is it that naysayers got wrong then and continue to get wrong today?

[Stephen Balaban]: The big thing is that cloud compute is not a commodity service. It is a very complicated, highly vertically integrated type of service that spans everything from land entitlement, construction, HPC, high performance computing, design, software, virtualization, cloud services on top. And there's a reason why the biggest companies in the world, these multi-trillion dollar market cap businesses, whether it's Amazon, Microsoft, Google, Oracle, are all in the cloud computing business is because it's a great business. And so I think that's like probably the fundamental thing that was misunderstood is that, oh, this is somehow a little bit different than a normal cloud service. But really what it was was it's a cloud service designed for the age of AI. But there is some element of commoditization right the price of rental of a GPU is going down but what you're saying is that to some extent it doesn't matter because it's only one layer of the cake.

[Matt]: Yeah. So when you look at for example I think it's like actually worth doing is to try to kind of dig into some of the methodology on for example an index like there's the index that's on Bloomberg for H100 rental prices. And what we're actually seeing in the market is that first of all there's two different rates. There's a public cloud on demand rate and then there's a long-term rental rate. And I think that some of these in indices don't properly take that into account because what we're actually seeing is a very consistent if not increasing long-term rental rate and very consistent and increasing on-demand rental rates. And so what happens is if the index mix for example if the methodology in the index biases towards long-term contracts being a bigger part of the volume that will look like a decline in the index when the reality is it's just a decline in the mix that the index is covering.

[Matt]: Fascinating. So I'm curious about your thoughts as a key leading player in the Neo cloud ecosystem about how you see the market evolve. How much of the competitive advantage that you guys are building and other players are building is based on technology versus financing race.

[Stephen Balaban]: There's a few different layers on it which is there's a lot of differentiation and work that's being put into for example the cloud software orchestration layer which allows us to for example take a very large scale GPU cluster and partition it up for our customers. So we've got, for example, our one-click cluster product that allows us to do that. And that's something that's like quite unique in the Neocloud space. Most of the other Neoclouds either don't have the ability to launch a cluster from their website or max it out at say 32 GPUs. Whereas Lambda's designed a piece of software that allows us to give you anywhere from 16 up to, you know, 4,000 GPUs in a web interface. And then there's innovation on the data center construction and design side of things which is also really important right because that's like the physical layer underneath the high performance computing equipment and you know we're working on a lot of different ways to dramatically reduce the time it takes to construct and stand up new megawatts. And then there's as you mentioned innovation on the finance side of things where you know we're coming up with new and unique ways to finance underwrite package these large-scale capital projects really and so I think it's like innovation's happening on every layer of the stack and it's a very complex coordination style business.

[Matt]: Yeah. And do you think that ultimately the NeoCloud ecosystem becomes a winner take all or is there room for multiple very large players?

[Stephen Balaban]: I think it's absolutely room for multiple very large players just like the traditional cloud business has shown that there's room for multiple large winners and multiple large players. And I think that the fundamental reason for that kind of going back to well what drives I guess market structure and I'd say generally speaking when you have an industry that has like technology modes and capital formation modes and economic modes that tends to be oligopolistic in its market structure. When you have markets that have more sort of network effect modes, those tend to be a little bit more, you know, single winner take all.

[Matt]: What are the various scenarios in your head as you think about the future about how it all play out? Are we overbuilding? Are we underbuilding? Nobody knows. How do you think about it?

[Stephen Balaban]: Well, I think that we continue to be generally underbuilding. And the most people that are sort of in leadership positions at NeoClouds or within the market have been recognizing this, you know, sort of insatiable amount of demand for large language models to do everything from, you know, being an assistant to code generation. You know, you can kind of look back to some of the talks that I've given in the past around I kind of called, hey, in a couple of, you know, months to years, we're going to be at a point in time where you can put money in and get software out the other end. And now at that point in time when you were predicting when I was predicting that, it was maybe not as widely held of a belief. But now with you know let's say the release of Opus 4.5 I think it's pretty clear that we have an amazing system that can take in money and output software and the I think the part which makes me feel so confident that there's going to continue to be demand is that we continue to see no end to the scaling laws which are like the underlying idea that you put more compute in and you get better intelligence levels out of your models. You know, as you increase the capacity of the model and train it with more compute, train it with more data, you get more intelligence out. And as long as that continues to hold, I think that we still have in store for us, it's hard to predict exactly when scaling laws might start to reach sort of a diminishing marginal return type of part of the curve. But right now it's very clear that we're going to continue to see more and more and more capable models that is kind of expanding the cone of the addressable market, right? Like originally the cone of the addressable market was all right, this is going to be helpful for customer support. It's a sort of substitute good for Google search and for other search online. And then now it's like well this is a substitute for a lot of software engineering roles or a huge augment to software engineering roles. And so as that cone expands the total market and the demand for compute expands and I think that we're continuing to underestimate it.

[Matt]: Do you worry about model training and model inference becoming I don't know 10x more compute efficient and what that would mean in terms of the buildup.

[Stephen Balaban]: I think that generally speaking what you're seeing is that if let's say you do become 10 times more efficient, I think that that just means that everybody is able to process 10 times more tokens and there's still the same fixed amount of compute in the world at any given point in time. And so in the early days, it's funny, we used to talk a lot about this back in let's say 2017. Oh, well, maybe there's going to be some new type of model, let's say, that will look more like a random forest model, which the audience might some members of the audience might know. You can kind of train a random forest model on a MacBook, right? And there was this concern that was kind of persistently raised around like, well, okay, what happens if you have this sort of adjacent disruption on the model side of things? And so far, we haven't seen that. And again, everything that we're building towards is sort of based on these scaling laws, which is really about scaling up this architecture. So, I don't really foresee a very likely outcome where we have this huge model disruption that would cause a decline in the demand for compute.

[Matt]: Where's the main bottleneck these days that you're experiencing building lambda labs? Is that GPU, power, electricity?

[Stephen Balaban]: So I always say that bottlenecks are always like kind of local before they're global in terms of you know one development might be bottlenecked on let's say generators or on UPS systems as a function of like the sort of idiosyncrasies of the site but broadly in the industry the thing that is the main bottleneck is basically land powered shell which is basically land that is entitled to have a certain amount of megawatt commitment from a utility and then of course the data center and the mechanical, electrical and plumbing equipment, the MEP equipment that goes into that data center. And so that's the main bottleneck that we're seeing in the industry right now I'd say across the board.

[Matt]: How real is the movement against data centers from the global community and how do you think about how to respond to it?

[Stephen Balaban]: Well, it's certainly it's like very popular in the news right now. I'd say that it's definitely very real. I mean, I think that rightfully communities that host any type of large capital project, whether it's a power plant or a solar farm or a data center or a distribution center, right? Those communities want to have a seat at the table. I'd say in general though, I spend a lot of time reading through a lot of the comments from communities and people want jobs. They want tax revenue. Any major capital development is going to bring a lot of tax revenue and it's going to bring a lot of jobs and it's going to bring investment into their community. And what they really are voicing I think is one is having a seat at the table while this stuff is being developed. I think that's an important thing just to have their voices heard and that the developers coming in and actually understanding the community. The other thing to kind of keep in mind is that there's a lot of misinformation out there. So for example, every single modern deployment of let's say a Blackwell class or a Rubin class GPU, you know, the VR GBN VR GPUs, these are often times in a closed direct-to-chip liquid cooling system that's connected to a dry cooler, which means that there's almost zero evaporation. It's not using evaporative cooling. It's using a dry cooler system that does not consume a lot of water. And on top of that, most of these data center developments are bringing a ton of power to the grid. They're either standing up behind the meter power. They're standing up and bringing battery electric storage systems to the grid. And they're bringing all these sort of ancillary benefits that strengthen and fortify the grid and also, you know, eventually in the long term will maintain the costs that are being experienced by the community. And so I actually think that there's a very clear path towards maybe spreading more of that fact around what does a data center bring because there's just a lot of misinformation. You'll see people talking about how data centers consume a lot of water. Well, an evaporative cooling tower might evaporate a lot of water, but practically no new builds in the United States are using evaporative cooling for doing these closed loop direct to chip liquid cooling systems.

[Matt]: Do you think we do a terrible job as an industry explaining this to the broader world because like those things keep coming back and they seem to be accelerating but then when you have the discussion there from a technical standpoint a lot of this is just simply based on museum information as you just said.

[Stephen Balaban]: I think that everybody's trying to get better at that kind of communication. And it just takes some clear thinking, writing down what are the benefits, writing down what are the costs and presenting that clearly and plainly to a community so they can make a good decision about, you know, what kind of jobs and what kind of development they want in their communities.

[Matt]: Let's open the hood for a minute. People talk about things like flops and GPU hours and tokens and MFU. what is the best way to think about a compute unit?

[Stephen Balaban]: Yeah, it's interesting. You know, you said a few different terms and I always like to kind of break it down from like a physics perspective into like the SI terms. So okay on the left hand side is all of the energy production and then on the you know my right hand side is sort of tokens being consumed by somebody and you know maybe you can even have the application layer on the far right of that that's using the token. So on the left hand side you've got either photons coming in per second or molecules of natural gas coming in per second and then that through a power plant or a solar farm gets converted into joules per second which is a measure of electrical power production and then the joules per second obviously in engines there's a level of efficiency and that's a engine efficiency. It's interesting because like the MFU percentage is kind of like an efficiency up on the higher end of that chain. The power plant or the solar plant then converts that into joules per second which is watts which is consumed by the entire data center. the data center itself needs to cool itself and that's the PUE and that's actually the efficiency metric that you can use to measure a data center on and then you put the servers and all the different networking and storage gear in and that's producing floating-point operations per second or flops per second. Okay, that is what gets consumed. The flops per second capacity is what gets consumed by let's say a model builder when they're training a model or when they're inferencing a model and that gets turned from flops per second into the tokens per second. Then on top of that tokens per second you might have some level of efficiency that the end customer is actually turning those tokens into real actual intelligence. That's like the entire pipeline I would say from end to end.

[Matt]: If two companies have the same chip fundamentally, how do they extract more value from it? What needs to happen to maximize the usefulness of that chip?

[Stephen Balaban]: If you look at the cost structure of let's say one GPU hour of time, you know, we're talking about H100s. The largest part of that cost structure is the depreciation that is associated with that GPU hour. And basically you can think of a utilization metric as being like kind of a multiplicative factor on that. So one over the utilization. So if you use your capital asset 50% of the time, you will have on a per hour basis twice the amount of depreciation expense associated with that. And so I think that the number one way that companies are gaining a unique advantage is, well, how can I build a cloud product that is beloved by people that is going to drive a high utilization. And in addition to that the market as we mentioned earlier for on-demand compute basically the retail pricing is obviously much higher than the wholesale pricing. So the retail is like on demand, spin up a GPU, spin down a GPU, normal cloud service. The wholesale is sort of buying 10,000 GPUs for 5 years for example. And so one of the things that we do at Lambda is really try to figure out, hey, how can we get the most dollar utilization and percentage utilization out of the capital deployments that we do and that's by making great cloud software that makes it easy for somebody to spin it up and down. So for example, if you don't have that cloud software, you can't rent you can't extract a retail pricing, right? you know, you cannot rent it out to somebody for an hour because you just simply don't have the means to be able to do that. And actually, a lot of NeoClouds are in that position where they don't even have the infrastructure to be able to run a real cloud service.

[Matt]: So, you have GPUs, but like a big part of how those data centers work is transforming GPUs into networks of GPUs. Do you want to explain at a high level how that works?

[Stephen Balaban]: The general idea is that you've got a large scale high performance computing cluster of a bunch of you know let's say Nvidia GB300 NVL72 racks that's 72 GPUs all worked together via Nvlink and then there's a connection between the racks that's either infiniband or high-speed Ethernet and that is essentially what's called the spine leaf topology which is basically a way to say hey this is a completely non-blocking every port on every GPU can talk with every other GPU in the network it's fully connected and it's able to provide maximum bandwidth between every individual GPU and that cluster is useful for training large models it's also useful for inferencing so frontier inference as we sometimes refer to it at lambda is basically very much a distributed inferencing problem where they actually will fragment or shard the model there'll be some sort of sharding strategy for the model where it can be essentially run on multiple GPUs and it uses that high-speed infiniband or Ethernet interconnect to do that communication.

[Matt]: And so what is a frontier inference is that in France for the most advanced reasoning models like the more demanding.

[Stephen Balaban]: Yes. Well, you know, it's not necessarily associated with reasoning models so much as like just a very large frontier model that is, you know, kind of the domain of let's say three companies in the world or four companies in the world when they're doing their inference. It's a very complicated thing that is fully utilizing all of the interconnection that's available.

[Matt]: And what you describe for Frontier in France is that conceptually the same thing as what happens for training. This concept of just distributing a task massively across a bunch of GPUs. What happens during a training run from a compute standpoint?

[Stephen Balaban]: Generally speaking, when you're doing a training run, you might think there might be some sort of split between the backwards pass and the forward pass on the model. And the backwards pass might be, let's say, 2/3 or more of the compute. And the forward pass, which is basically the same thing as inferencing, is the remainder. And one of the realizations that I think has been made over the last bit of time is that the type of infrastructure that you'd want for doing a large scale training run can be reused to do the inferencing of that model and what I mean by the sort of frontier inference and the fact that the inferencing is being done in a distributed way you know you'll have like a mixture of experts model and there'll different basically sharding strategies for how you put those experts onto different servers and to different GPUs. And you know the models can be very large. They may not fit on one single rack or you know they may not fit on one single server. They might need to be distributed across different servers to even just do the forward inference pass. And so that's where sort of distributed frontier inference kind of comes into the picture, right? Because like if you're doing a small model, let's say llama that the users might be familiar with or some of the quantized small models can fit on a single GPU. Okay. Well, let's just say that like Opus and Chad JBD 5.5 can't fit on a single GPU.

[Matt]: And when we think about compute costs, what costs the most money? Is that model size? Is that memory bandwidth? Is that latency? Does context window and like those very very large context window do they change anything to the compute cost?

[Stephen Balaban]: As I mentioned, like the biggest component of the unit cost for a cloud service like this is the depreciation expense. And within that, is basically some sort of bill of materials for the servers that are in the data center, which is by far and away the biggest portion of the cost. If you talk about the capital stack, let's say you can go back down to power generation, 2 to 3 million a megawatt. 2 to 3 billion a gigawatt for a power plant. The data center is between 10 and 15 billion a gigawatt for building the data center. And then the compute the servers can be anywhere from 35 to 45 billion a gigawatt. So you can see the server portion is obviously by far and away the largest and that's like a big part of the depreciation expense and then within that obviously you have the server and cluster bill of materials which is primarily the GPUs. If you were to kind of break down Nvidia's bill of materials, then you can kind of get better allocation towards where those costs are coming from. But certainly in the most recent period of time, memory expenses, you know, memory has gone up a lot in price and there's very few vendors, right? You know, for HBM memory, but Samsung, Hynix.

[Matt]: So, you guys are a big Nvidia shop at a precise level. You mentioned some of the names, but like which chips do you use mostly? What's your kind of chip stack?

[Stephen Balaban]: Yeah, so Lambda really loves Nvidia's products. I mean, they're the only server provider, the only chip provider that is available in every single major cloud platform, which is a huge platform advantage. And we've stuck with the Nvidia sort of ecosystem for all the chips we've deployed. And we've got everything from V100s, A100s, H100s, H200s, B200's, GH200s or GB200, B300s, and VR200's coming soon. And so we use everything in the ecosystem.

[Matt]: Do you think that today or in the near future we're going to be in a multi-silicon kind of world? Is there like room for different players beyond Nvidia?

[Stephen Balaban]: Well, I mean, I think that we're already in a world where there's a huge amount of competition from massive massive multi-trillion dollar companies and they're all trying to fight for the same thing, which is to be the best chip in the world for running and training neural networks. Essentially, Nvidia's built a great product that has gotten a lot of distribution and has a great platform of developers who love what they do. And you have to take into account not just the cost of the chip, right? The price of the chip is one aspect, but you know, you have to take into account the entire software ecosystem and what's been developed. So, one of the big people talk about like what's Nvidia's moat? One of the big moats they've got is just the CUDNN stack. It's not just CUDA. You know, CUDA is sure that's like the water we all swim in, but like CUDNN has got so many matrix multiplication routine optimizations baked into it.

[Matt]: What is CUDNN for everyone to understand?

[Stephen Balaban]: So, CUDNN is the CUDA deep neural network library and it's basically Nvidia's you can think of it like a highly tuned engine for matrix multiplication. And basically, if you were to just sort of naively implement the matrix multiplication algorithm, you would maybe get a certain level of floating point operations per second. But they've gone and tuned every single aspect of it and come in and do Winograd filtering or a bunch of different algorithms that you would apply to speed up matrix multiplication and CUDNN means that you don't have to go and do the optimization yourself and so that's one aspect the other one is NCCL which is their networking optimization library where it will sense the topology and the connected nature of your network, your Infiniband or your Ethernet network and it will suggest an optimized routine for doing reduce all and broadcast the different what are called open MPI primitives which are used for that sharding that we were talking about for both training and for inference. And so that's the kind of software stack that I think really is hard for a lot of the new entrants in the chip space to overcome. I think we're already, like I said, we're already in a world where there are multiple options for silicon. You know, the biggest labs in the world are using multiple different types of chips to do their inferencing and training on.

[Matt]: What would be a plain English definition? We talked about the chips, but like the rest of the stack, the networking and the storage. Just walk us through how it works.

[Stephen Balaban]: When you're running a cloud service, you'll train your model or you'll upload your trained model and you're ready to start doing large scale inferencing. Well, you're going to need a place to put your data whether it's the data that you're using to train with or whether it's the data that's coming in and streaming in from your end customers. And so having high-speed storage is like a really important part of it. So, Lambda offers the basically AI optimized file system service that is significantly faster than like your standard let's say cloud file system which is maybe more of a traditional NFS type of thing. This is like a highly optimized parallel file system that's designed for high performant read and writes and mostly high performance reads. That's like the kind of most of the workload.

[Matt]: And that's something you build in-house completely.

[Stephen Balaban]: We have I mean well in-house completely right it's like you have to ask the question like what is the definition of in-house completely right you know like we've never spun a PCB at this company we have not authored for example you know we use KBM for our virtualization for example right and so you know what we have both commodity off-the-shelf hardware that has software installed on top of it for our some of our storage we have some storage partners that we work with as well. But you know generally speaking everything that we do on the cloud I would generally say is something that is like we rolled it ourselves with the help of the broader ecosystem because again there's no such thing as rolling it yourself unless you're like mining ultra pure silicon and then coming up with your own ASML.

[Matt]: Yeah, that's the highly optimized storage. What else? The networking part and what other pieces?

[Stephen Balaban]: So, I was talking about this one cloud cluster product that we've got and the way just for everybody to think about this is like okay well look you've got a bunch of GPUs. Let's say you've got a cluster of 10,000 GPUs. Well, I want to partition that cluster up. And so what it is is it's a bunch of GPUs. some CPU servers as well because you need to have an orchestration fleet as well and then you've got some storage and all of the CPU servers and the storage servers and the GPU servers are interconnected with the storage so they can quickly read and write from it and that communication happens over what's called the inband network and then there's the compute fabric which is where I was talking about where all of the sort of weights and feature activations are being shared throughout that compute fabric and then there's an out-of-band monitoring network where you've got access to whether it's BMC or some of your DPUs and when you are trying to create a subpartition of a 10,000 GPU cluster you need to simultaneously partition the inband the out of band and the compute fabric okay so like that complex coordination between we've got a bunch of bare metal systems to hey we've got a virtualized system that has what's called RDMA, remote direct memory access that allows them to read and write quickly not just from the disks but from each other's memory the GPU's HBM memory and allow them to do that sort of direct memory access allowing it to go directly from a GPU to another GPU without getting copied to the CPU for example. Having that all work is an immense software undertaking and this is going back to the original question like well what are people not getting about Neoclouds? Well, first of all the answer is that most Neoclouds don't have this kind of technology. Most Neoclouds have not made the really it's like kind of high tens to hundreds of millions of dollars of software investment that you need to make to build a real cloud system that can partition a high performance computing environment like this and then to have it all work with the storage. Anyways, I guess that sort of summarizes the steps that you need and kind of you can think about all the different moving parts of a modern like how does an AI data center work? People talk like AI data center, but really you have to kind of go down that one next level down, which is because if you were to, by the way, if you were to ask an AI data center landlord, a traditional one, what's going on inside of the data center, they'd be like, well, look, we're real estate people, and we really outsource this to the GC, but the GC doesn't know, of course, anything that's going inside. And this is then it's their tenants who know. So this is what's actually happening inside of an AI data center and then it serves the result like also going back to the community stuff. If people knew a lot more about like well this AI data center is actually just serving the chat GPT request that I'm giving it, right? Like sometimes they don't even realize that that's actually what an AI data center does.

[Matt]: So you mentioned tenants. Do you rent them? Do you also own some building some? and where does that fit in the overall strategy?

[Stephen Balaban]: Yeah. So, initially we started off as being primarily a renter and we've actually started to get into the business of financing some of them the construction of them ourselves as well as you know we're going now into full vertical integration where we are identifying land coming to the table with a basis of design which is basically all the engineering diagrams to construct the data center financing and constructing that data center, putting the servers in, and then associating that with a long-term offtake agreement with one of the major compute consumers in the world and financing it all. So, like we're getting into full vertical integration at Lambda and it's been great because we've been able to kind of again bring that engineering mindset to this problem which was historically mostly run by people in real estate.

[Matt]: In your own data centers, are you the sole tenant or is part of the idea that you can also rent some to others?

[Stephen Balaban]: In a lot of our data centers, we are the sole tenant. In terms of the data centers that we're planning on constructing, we don't yet have any plans to lease that space to others. So, we're not trying to get into the leasing data center business. Maybe that's something that you can imagine down the road. I wouldn't write it out completely, but for now, we have to focus on providing Lambda with the compute that we need to service the market.

[Matt]: How international are you, by the way?

[Stephen Balaban]: You know, I'd say that we're very much focused on North America. And so we have data centers in Canada, United States, and Mexico. We're very much, like I say, primarily focused on North America, but really within, you know, that the United States obviously. And we haven't had this desire internally to try to go and expand into Europe or too far into Asia. We've done some partnerships with some of our great investors like SK Telecom and we have a data center that we've operated in Korea in Seoul. And so we have some experience with international but right now we're just like look let's focus on the US market. It's where the opportunity is.

[Matt]: Do you need for performance reasons to be close to the customer the way you need to have regions in cloud?

[Stephen Balaban]: You know, it's super interesting. A lot I get this question a lot and people they're like well does latency matter does so I'll tell you what matters and what doesn't matter. You can look at your own utilization of whether it's chatgpt or claude or grok or gemini and you can see hey a lot of the things that I'm doing I kind of shoot it off I come back later and there's a research report for me maybe it's a long-running agent workflow in those cases latency doesn't matter at all the only thing that matters is your cost per token that's all that matters. So that's been a really interesting change. I think that the old school traditional legacy cloud business was so latency focused because of some of the applications but this new fleet of AI applications are far less latency sensitive. So that's one. But there is the caveat which is this governance and data governance is becoming an important thing and a lot of countries are wanting to have the AI compute that their citizens are using be run out of their own country so that they can at least have their own perception of control or whatever and that is another element to it but I'd say that from the latency there's no technical reasons.

[Matt]: Let's talk about the financing stack. So presumably it's a commission of equity and and debt. How does it all work?

[Stephen Balaban]: Yeah. So the way that it works is that you could really fragment it into these two parts which is like financing your on demand cloud versus financing an offtake agreement which is like a longer term commitment. And on the on-demand cloud you're kind of looking at Lambda's credit quality. on the offtake agreement you're kind of looking at the credit quality of the end customer who's paying the bill and so what you do is you just take your offtake agreement you take this chunk of GPUs that you're deploying you take a lease or the property and you kind of put it into a box and you can go to the private credit markets and you can come up with an asset-based loan you can get a variety There's a variety of different methodologies for financing it. Most of it is just some sort of special purpose vehicle that's designed to finance this particular deployment with a very known and easy to underwrite which is basically just a fancy way of saying the finance term for just assessing the risks and the downsides of a particular credit investment. And there's a vibrant private credit market for that. On the on demand cloud side of things, it's not quite as mature as when there's a for example an investment grade offtaker agreement. But it's becoming more and more mature and in general creditors and lenders are really starting to understand the value of an Nvidia chip because, you know, you actually look at the chips that we deployed in 2023, H100s, we're now leasing those out at a higher rate now than we were originally in 2023. So, these creditors are starting to look at these assets and say, "Wow, this is an asset that is very valuable and also easy for us to underwrite." And of course, while they are underwriting towards the actual cash flows that are coming out of that agreement, just as an asset class overall, people are realizing that this is a really great opportunity. And so, creditors are starting to flock to these deals.

[Matt]: You're renting an H100 at a higher rate because why? because the demand for compute is so rapid that people will take any or the technical depreciation of the product is slower than people thought. What drives that?

[Stephen Balaban]: Well, what's driving it? I mean certainly it's the demand being high increases the price that you're able to get in the market. There's no question about that fundamental law. Again, going back to what people didn't understand about this market, there was people who were saying, "Oh, well, there's a five-year lifetime or three-year lifetime." I even heard some people say three year lifetime for these GPUs. This completely false. You know, we have GPUs that we've commissioned and we're one of the earliest Neoclouds. In fact, we're probably the only Neocloud that actually has GPUs in our fleet that are fully depreciated from an accounting perspective, right? which is most people are adopting around a six-year accounting depreciation schedule. But that's not the usable life. The usable life is longer than the accounting depreciation schedule. And what really matters is the economic usable life. And so what we're starting to see is that the people who are the naysayers, oh this is going to be you're going to throw these GPUs out in 5 years are completely wrong. They're completely wrong and they've been wrong the entire time.

[Matt]: Do you think there is going to be or do you already see happening some kind of financial market for compute units you know with trading and derivatives is that happening

[Stephen Balaban]: I'm starting to see some people start to examine what a maybe vibrant spot market you know first you need to have a spot market for something before then you can establish a derivative like a future or other more exotic things. I'm starting to see that. But fundamentally, I think that the asset class is just starting to mature and creditors are starting to become very comfortable with investing in the credit side of buying Nvidia GPUs and deploying them into data centers and we don't need to get too fancy with it. That's kind of like my part of my opinion is that I think that the market is starting to mature that may be an eventuality is having more complex securities that surround GPUs. But I think for right now people are starting to realize that it's a great credit investment and that's what's changed I'd say over the last year is that people have started to really treat it like a more mature asset class.

[Matt]: Maybe quickly just go back to the very origin because I think you've been in the effectively in the AI world the whole time but are coming from a very different angle with multiple pivots. What did you start with and when?

[Stephen Balaban]: Well, you know, with the complexity of the business, you can now, you know, the complexity, the capital intensivity, just the sort of not fitting into a box and you can see why we've oftentimes not had a lot of traditional venture investors in Lambda. and all of our investors have done exceptionally well but they've kind of come from more often than not outside of traditional let's say mainline Silicon Valley VCs. So just going back to the origin story I started Lambda in 2012 and we were a facial recognition software company so I was training convolutional neural networks to do face and image recognition And we eventually hosted that on an API. I was training those convnets on a 4X Nvidia GTX 580 workstation that I had bought from a friend who had built it actually. And this was, you know, really pretty avant-garde stuff at the time. Most people didn't really believe in what was called the field called deep learning at the time. And that was inspired by the imagenet 2012 moment or that was even before that the Imagenet moment. You know I pulled the CUDA convnet repo off of Google code. That's how you know how old Lambda is is that Google code was still around and I pulled the CUDA convnet codebase and was like playing around with it. I got very lucky that the AlexNet paper had been published the same year that Lambda was founded. It's not a coincidence at all. We launched this space recognition API, got a couple thousand users, but it wasn't really generating a ton of cash. And sort of as part of that the complex story of startups, you know, in parallel, I was sort of I found these guys who had just graduated from their PhD programs the gentlemen Zach and Nico and they had said, "Hey, we're going to start a company." I said, "Hey, you know, let me help you guys out. I'm going to work with you for a year. I'm going to learn a little bit more about neural networks." And I helped him out on this company, helped them get a company called Percepio started. And I was the first employee there while I was running Lambda. And we were running these convnets locally on the iPhone. And again, this is 2013. So we were using the GPU image library and just straight open GLES shaders like the shaders that are used for rendering. We were using those to run the convnets on the iPhone and eventually I kind of left to go continue to work on Lambda full-time and probably about a year or so later they got acquired by Apple. And so if you know the feature on your iPhone where you swipe up on an image and you can recognize faces and search through your library, that's maybe some of the stuff that eventually got integrated into iOS through that acquisition. And then Lambda, we continued on. I had we had a variety of different products. Everything from Lambda hat, which was a baseball cap that took a camera every 10 sec took a picture every 10 seconds with a camera embedded in the tip of the brim for gathering data sets for image and face recognition.

[Matt]: which is fascinating because fast forward to today and that's a whole segment, right? Like capturing everyday life to train the AI.

[Stephen Balaban]: It goes to show you have to, you know, it's one, it's important to be able to see the future. It's also important to get your timing right as well. Right. And now it all worked out, right? Despite maybe that Lambda hat product not being great, but it taught me a lot about how to build hardware. I lived in Shenzhen for a little bit, working on the PCB and spinning the PCB and designing the actual hardware product. And, you know, it taught me how to make consumer electronics. And that was actually a huge huge skill because it totally opened my mind to new ways of doing business that aren't just making apps, right? And eventually we had this product called Dream Scope which became really popular in 2015 and 16 and it was basically using the Google Deep Dream methodology of using a convnet to generate images. It's like an early version of midjourney or whatever. and DeepDream and the Leon Gate style transfer algorithm allowed you to turn a photo into a painting basically. And we got like a million users on that, processed tens of millions of images, maybe 15 million images or something like this. And that caused us to have a huge AWS bill. It was like $40,000 a month or something. And so to replace that, we'd ended up building a little cluster out of workstations. And then that was a $60,000 capex that we were terrified to make, by the way. We were so scared that doing this capex was going to put us out of business. We made it out of workstations because we thought, oh well, worst case scenario, we can just sell them. And so lo and behold, we did end up turning it online and it brought the bill down to zero. So it paid itself back in a month and a half. And we thought, wow, this is like we're saving more money than we're making. Maybe we should be in the business of providing compute to other AI researchers. And thus, we started selling workstations and servers and started developing our cloud platform. Maybe did $3 million of revenue in 2017. That first year, selling workstations, then 10 million in 2018, then $30 million in 2019. We grew the hardware business over the next couple years to probably about $200 million run rate. And then the cloud business we really started in 2019 and we started development before then but we started really marketing it and it kind of was slow to grow to be honest because not a lot of people in 2018 and 19 and 2020 wanted a bunch of AI compute. There was a pretty niche market for it. But eventually our cloud business continued to grow and now it's at a little bit under a billion dollar revenue run rate. we've fully exited the hardware business and so yeah, Lambda's got a absolutely wild founding story to summarize.

[Matt]: Are some of the people that were there at the beginning still around? I think you started the company with your brother. Is that right? And your brother is still at the company.

[Stephen Balaban]: Yeah. And so in terms of like the early people basically it's not I mean not even basically of the four people who are making Dream Scope me, Michael Balaban, my co-founder and fraternal twin brother, Tran Lee who's our chief scientific officer and then Steve Clarkson who's an engineering leader at the company and has a bunch of folks reporting into him. Now they're all still at the company. The next hire, one of those gentlemen named Mateesh Agaral who's one of the next hires in that team. He was with the company for maybe eight years or something like this. and then he eventually left and joined another former Lambda team member Thomas Summers to start Positron which is an accelerator company and they're like now valued at over a billion dollars. So not only has the original team stuck around but we've already started to kind of see what a lambda alumni a lambda mafia network looks like in the world.

[Stephen Balaban]: 当你经营一家如此资本密集、营运资金密集的初创公司时，随着你的成长，系统会受到很多冲击。例如，在 2020 年 4 月，软件公司感觉很好，因为他们可以交付软件，需求更大。而硬件公司，码头关闭了，你在 3 月和 4 月无法确认收入。我清楚地记得所有这些事情。我记得站在团队面前说，“嘿，听着，现在非常艰难。我们不确定是否能挺过去，但唯一能做的就是咬紧牙关，享受痛苦，冲过去，为眼前的问题找到解决方案，所有这一切都是为了取悦客户。” 因为从根本上说，最重要的是让大家团结在同一个目标上：我们在这里的唯一原因就是构建人们想要、喜爱并会告诉朋友、愿意付钱的东西。其他一切都源于这种取悦客户的体验。例如，在做入职培训时，我过去常常在 Lambda 101 中展示一张 Linux 企鹅的图片，它坐在 Lambda 工作站上，正在阅读 GPT-2 论文，训练有一个损失曲线。我说，“把自己想象成那只正在使用这个工作站或云服务训练神经网络的企鹅，想想什么能取悦它。” 无论是我们发货团队的人说，“嘿，我们在盒子里放一些 T 恤吧。” 所以每个工作站都附带一件 Lambda T 恤。或者数据中心运营团队的成员说，“嘿，我们应该用白色机架，因为那会让我们与众不同，让一切看起来很棒，我们会很自豪地展示它。” 这些就是你向公司灌输“客户至上”理念的方式，我认为这能帮助你度过难关。

新任 CEO 与未来愿景

[Matt]: 最近的一个演变是，你们请来了一位新 CEO，我的法国同胞 Michel K，来管理公司。请谈谈你们的想法，是什么促使你做出这个决定，以及这将如何为公司下一阶段做好准备？

[Stephen Balaban]: 作为一个创始人，能够走到公司有能力引进像 Michel 这样优秀的人才担任这个职位，是一种巨大的荣誉。因为很多公司，创始人 CEO 可能出于 ego 而必须担任 CEO。我个人从未有过这种感觉。我在乎技术，在乎打造一家伟大的世代公司，而且我认为有很多不同的职位可以实现这个目标。所以，当我们成熟到能够负担得起像 Michel 这样的 CEO 时，他之前是软银国际 CEO、Sprint CEO、阿尔卡特朗讯 CEO，还在一些非常棒的公司董事会任职，包括迈凯伦。我过去做融资、资本形成和日常业务管理是出于必要，而不是因为我真的喜欢做这些。我认为有很多创始人 CEO 热爱他们工作的每一个方面，但私下里，当我与创始人 CEO 交谈时，我总是问，“你有多讨厌它？”

[Matt]: 我觉得人们不觉得整天和 VC 交谈很兴奋，这很令人震惊，但我相信你的话。

[Stephen Balaban]: 对我来说，能够围绕公司组建一个团队，看到每个人都在他们喜欢专注的事情上蓬勃发展，这是一次很棒的经历。例如，现在我作为 CTO，主要关注的事情之一是公司的高速数据中心部署是什么样的。我希望 Lambda 成为这样一个垂直整合、高速度的强大力量。当你放眼世界，你会发现世界上只有两家公司能够进行高速部署：SpaceX 和 Lambda。我们极度专注于如何削减流程中的每一个小环节，以更快地部署计算。这就是我作为 CTO 一直在深入研究并真正享受的事情。

[Matt]: xAI 在启动时的记录是怎样的？

[Stephen Balaban]: 我想大概是 200 多天。

[Matt]: 是的。你认为这个记录可以以可重复的速度被匹配或超越吗？

[Stephen Balaban]: 我认为可以匹配或超越。是的。

[Matt]: 这主要是流程问题。

[Stephen Balaban]: 我认为这涉及到方方面面，从选址过程、选址时使用的约束条件、MEP 管道、数据中心建设方式，到如何让最终客户消费这些计算，以及如何从流程中削减很多东西。因为很多时候，设计这些数据中心的人，就像我提到的，是房地产人士，他们被超大规模云厂商揪着脖子说，“去建这个设计，去找个总承包商干活。” 他们对里面是什么一无所知。另一方面，超大规模云厂商一直在为传统的云服务建设。如果你看看任何云厂商的现代区域，它们有数百种服务，从卫星基站到磁带存储、旋转磁盘、人脸识别 API。每一种服务都需要不同的配置和参数。你可能有人在上面运行 ATM 后端，这与 AI 数据中心的设计空间和约束条件截然不同，AI 数据中心可能对可用性和正常运行时间要求较低。所以，我认为这正是 Lambda 能够通过这种有针对性的 AI 优先方法建立大量独特价值的地方。

神经软件与自组装软件

[Matt]: 你曾说过一句话：AI 不会编写软件，它将成为软件本身。你这是什么意思？

[Stephen Balaban]: 这是我关于所谓“神经软件”或“神经操作系统”的想法。最好的体验方式是去你的 ChatGPT 或 Claude，说，“嘿，给我渲染一个 ASI 艺术桌面界面。” 你完全在文本领域工作，我希望你假装成一个操作系统。我会说点击这个，打开那个，你就像一台计算机一样运行。给它那个提示。我认为你会看到，大型语言模型成为软件本身，而不是生成软件的未来。这导致了一种极其灵活和可塑的与计算机交互的方式，其中不可能有 bug，只有对提示和你所要求的内容的误解。我认为，对于你电脑上的许多软件，你可能会看到这种情况开始接管。你可以通过 ASI 艺术一窥未来，最终它还会有一个多模态网络，生成屏幕上的每一个像素以及扬声器发出的每一个音频波形。这样做的好处是，你可以真正地构想出软件。只有你正在体验的部分才被实际实现。它可以拥有你要求的任何功能。我认为这是一种非常强大的与计算机交互的方式。

[Matt]: 所以，不是你给 LLM 简单的指令，而是 LLM 本身就是软件。

[Stephen Balaban]: 打个比方，Vibe Coding 接收一个提示，然后输出人类可读、可写、可编译的代码，这些代码运行在正常的人类软件编程语言基础上。它输出 C 代码，通过编译器；输出 Python 代码，通过解释器。那个软件是静态的，一旦生成就不能改变。你可以再次进行 Vibe Coding，或者即时 Vibe Coding。在传统人类编写软件和 Vibe Coding 软件之间有一个渐变过程。然后你进入即时 Vibe Coding 软件，它是软件应用的实时创建。但这仍然是软件。再下一步，你直接与 LLM 交互，它模拟软件可能的行为方式。这就是 Vibe Coding 和神经操作系统或神经软件之间的区别。神经软件没有运行的代码，它只是神经网络思维中特征激活空间和上下文的修改。

[Matt]: 你认为我们离那还有多远？那是……

[Stephen Balaban]: 我们今天已经有原型了。

[Matt]: 当你说“我们”时，是指 Lambda 吗？

[Stephen Balaban]: 是的，Lambda 开发了一个原型。还有其他多家公司也开发了原型。也有学术研究概述了这可能是什么样子。至于大规模采用，一般来说，当我对某件事过早下判断时，我往往会早十年到十五年。所以，我认为神经软件将在十年到十五年内开始或实现大规模采用。实际上，你已经有了一个例子：特斯拉的自动驾驶汽车，或者任何类型的端到端神经网络和大型模型进行自主驾驶，都是神经软件的一种形式。人们理解那个方面，它看到视频，做出关于输出什么的决定，用户体验就是驾驶体验。我认为这是一个神经软件的例子。我们今天已经看到了。现在的问题是，每个人的电脑何时会采用它？我猜是十年。

Agent 的影响与内部使用

[Matt]: 从你作为计算提供商的角度来看，Agent 会改变什么吗？如果会，以什么方式？

[Stephen Balaban]: 要理解计算层需要改变什么，我们需要理解用户正在改变什么。当你使用 Agent 进行 Vibe Coding 时，你会注意到，你的挂钟时间大部分花在运行测试、收集数据、搜索代码库上。很多时间实际上不是花在推理神经网络上，而是花在做其他事情上。这很像软件工程师如何分配他们的时间。还记得 XKCD 那个关于编译的漫画吗？他们在办公椅上剑斗，有人问“你们在干嘛？”“编译。” 所以现在有很多时间花在编译上，很多时间花在运行测试上，因为 Agent 24/7 循环工作得好的方式，是你不断地用一套很好的自动化测试来验证你写的代码是否正确。那么这意味着什么？这意味着每一个云服务都需要开始做更多的传统 CPU 工作负载。它们需要专注于提供一个伟大的、安全的环境来托管你的编码实例。然后你需要从安全角度思考，如何保护这些大量涌入的新应用。

[Matt]: 你们内部是如何使用 AI Agent 的？

[Stephen Balaban]: Lambda 的很多工程师已经在使用完全由 Agent 驱动的工作流了。你只需去 Cloud Code，说使用高级工作流或启动 Agent，就可以做到。这是第一步。我在内部演示过，一些人已经采用了所谓的“自组装软件”。自组装软件的想法是，你将产品需求和来自系统的持续用户反馈，连接到一个 24/7 运行的 Agent 集群。这样就形成了一个非常清晰和紧密的循环：提交一个 bug 或功能请求，就有一群 Agent 为你实时实现它。我把这种循环称为自组装软件，因为你定义了软件的用途，但大部分开发工作是在软件发布后，用户开始与之交互并集体为其定制时才发生的。我认为这可能是未来很多 Agent 驱动开发的发展方向。另一方面，一旦模型变得更智能，虽然还没完全到那一步，但 Agent 会说，“嘿，我需要人类帮助我。我需要你为我插上一千块 GPU，或者给我一个特定服务的 API 密钥，或者去帮我注册个东西，你能去谈判一下吗？” 我认为这就是你将开始看到的情况：产品用户反馈由 Agent 实现，然后 Agent 也会要求公司里的人为它们做事，所有这一切都是为了取悦客户和赚钱。

吉瓦级工厂与一人一 GPU

[Matt]: 你之前谈到过吉瓦级工厂。这是你之前描述的，比如非常擅长快速创建数据中心，但也让它们变得更大。这个概念是什么？它是一个 AI 工厂，基本上是土地、数据中心、里面的服务器，生成 Token。吉瓦级意味着它消耗一千兆瓦或十亿瓦特的电力，这很大。作为参考，纽约市大约是 5 吉瓦。你还谈到了“一人一 GPU”。这是你对未来的愿景吗？请为我们解读一下。

[Stephen Balaban]: 在人们真正相信 AI 理论之前，我在 B 轮和 C 轮融资时，经常谈论计算机行业和 AI 行业之间的相似之处。我真的觉得 AI 正在形成一批世代公司，随着 AI 带来的变化，将会诞生一批世代公司。那是在 2020、2021 年。如果你读一下苹果公司的历史，早期的座右铭和信条是“一人一电脑”。在“一人一 GPU”中蕴含着一种谦逊，它源自“一人一电脑”。想想史蒂夫·乔布斯是多么有远见。苹果公司成立于 1976 年，Macintosh 在 1984 年推出。那是成立 8 年后。那时实现“一人一电脑”了吗？不，差得远。1984 到 1994 年，互联网刚刚开始繁荣，还没到。2004 年，我们终于有了宽带互联网接入，在美国，可能第一次接近“一人一电脑”，但更像是“一家一电脑”。直到 2014 年，40 年后，才可能真正实现“一人一电脑”，甚至超越了它，因为人们有笔记本电脑和手机。最终，直到 2024 年，苹果公司成立大约 50 年后，电子商务才开始真正渗透。我之所以选择“一人一 GPU”，是因为我相信，在未来，美国的每个人都需要一块或更多 GPU 的计算能力来完成日常工作、享受生活，无论是获取信息、娱乐、提高生产力还是发挥创造力。同时，我也认识到，史蒂夫·乔布斯和苹果，作为资本主义历史上最优秀的公司之一，花了半个世纪才实现他们的目标。所以，我认为这不是一个可以一夜之间快速实现的目标。这就是它对我的意义。

快速问答：过热与低估

[Matt]: 最后，准备好回答几个快速的热点问题了吗？

[Stephen Balaban]: 当然。

[Matt]: AI 中哪个想法被过度炒作了？

[Stephen Balaban]: 我认为很多针对非软件工程领域的 Agent 工作流被过度炒作了。原因是，让 Agent 工作流运行良好的一个方法是，它需要有非常具体的反馈机制，这在自动化测试中效果很好。但对于像“去买个网站”这样的任务，效果就不好，因为没有牵引力让模型在长时间内迭代。所以，我认为对于不易验证的事情，Agent 工作流被过度炒作了。但我也不是说所有非软件工程领域都这样，因为有很多容易验证的领域，比如 CAD、计算机辅助制造、有限元分析、计算流体动力学，在这些领域你可以很好地运行 Agent 工作流，进行模拟和迭代。但对于“Claude，给我赚十亿美元，别犯错”这样的任务，情况就不同了。

[Matt]: 通货膨胀会怎样？

[Stephen Balaban]: 那不会是通货膨胀，实际上只是经济中的价值创造，甚至是通缩的。

[Matt]: AI 中哪个想法被低估了？

[Stephen Balaban]: 我确实认为神经操作系统和自组装软件的一些方面被低估了。有趣的是，我会给出同样的答案：用于软件开发的基于 Agent 的工作流。我认为大多数人并不理解，他们真的不理解，因为他们从未尝试过。他们从未去过 Cloud Code，说“最大努力”，使用最新的模型，然后去构建任何他们想构建的东西，并启动 10 个 Agent 去做。我认为很多人仍然没有这样做过。

[Matt]: 好了，Stephen，这真是太棒了。非常感谢你抽出时间与我们交流。

[Stephen Balaban]: Matt，非常感谢你的邀请。

[Matt]: 很荣幸。

[Matt]: 嗨，我是 Matt Turk。感谢收听本期 Mad Podcast。如果你喜欢，如果你还没有订阅，我们非常感激你能考虑订阅，或者在你观看或收听本期节目的任何平台上留下好评或评论。这真的有助于我们建设播客并邀请到优秀的嘉宾。谢谢，下期节目再见。