The good news is infrastructure is sexy again. So that's kind of cool. This is like the combination of the buildout of the internet, the space race, and the Manhattan project all put into one where there's a geopolitical implication of it. There's an economic implication, there's a national security implication, and then there's just a speed implication that's pretty profound.

I mean, I think it's easy to say. I've seen nothing like this. I'm fairly certain no one's seen anything like this. The internet in the late 90s, early 2000s was big and we felt like, oh my gosh, can't believe the uh buildout, the rate. This makes it I mean 10x is an understatement. It's 100x what the internet was.

Hello. Hello. All right. What better time and place to talk infrastructure? All right. So, we were back in the green room and just as um the first question was getting answered, I got cut off. So, this could be an entire repeat for all I know. So, but anyway, let's go. Right. Um the first question is similar. So, both of you firstly welcome and thank you for being here. and hope you'll have a great day and a half as well. Um, both of you been in the industry for a while and both of you have lived through many infrastructure cycles, right? So, have you seen anything like this cycle from your vantage point? Not from an investor vantage point but from your internal um vantage point where you are responsible for building things and and planning for things and so on. Anyone of you? Where do you want to start? You want to start a mean?

I mean, I think it's easy to say. I've seen nothing like this. I'm fairly certain no one's seen anything like this. The internet in the late 90s, early 2000s was big, and we felt like, oh my gosh, can't believe the uh build out, the rate. This makes it I mean 10x is an understatement. It's 100x what the internet was. Um I think the upside is as big as the internet was, same thing. 10x and 100x. Yeah. No, nothing like it.

Yeah, I'd agree. I don't think there's any priors to this size, the speed and scale. Um I' I'd say um the good news is infrastructure is sexy again. So that's kind of cool. Um it was a long time or it wasn't sexy. Um the um the thing I would say that's that's really interesting is this is like the combination of the buildout of the internet, the space race and the Manhattan project all put into one where there's a geopolitical implication of it, there's an economic implication, there's a national security implication, and then there's a um you know just a speed implication that's pretty profound. So uh yeah, we've, you know, none of us have ever seen it um at this size and scale. On the other hand, um I think we're grossly underestimating like there's the most common question I ask right now is is there a bubble? I think we're grossly underestimating the buildout. I think there's going to be much more needed than what we are putting the um you know projections towards.

So that's the following question is where are we do you think in the capex spend cycle? But more importantly, what are the signals that you guys use internally right in your thinking? I mean, you have to plan data centers whatever four five years in advance. You have to buy nuclear reactors and whatnot. So, how do you think about the demand signals as well as your technology signals and G2 same thing for you, but from the point of view of enterprise and neo clouds etc.

uh we're early in the cycle is uh what what I would say certainly relative to the demand that we're seeing our internal users are uh we've been building TPUs for 10 years uh so we have now seven generations in production for internal and external use our seven and 8y old TPUs have 100% utilization and that that just shows what the the demand is everyone would of course prefer to be on the latest generation uh but whatever they can get so this tells me that the demand is tremendous, but also who we're turning away and the use cases that we're turning away. It's it's not like, oh yeah, that's kind of cool. It's, oh my gosh, we're actually not going to invest in this and there's no option because that's where we are on the list. Same with many of you in the room, right? We're we're working with many of you in the room and many of you are telling me directly and thank you. Um, we need more earlier, right?

Now the challenge here though is as you said that we're limited by power, we're limited by transforming land. We're limited by um permitting and we're limited by uh backup delivery of uh lots of things in the supply chain. So one worry I have is that uh the supply isn't actually going to catch up to the demand as quickly as uh we'd all like. I I heard the previous session some of the discussions of the um trillions of dollars that we're going to be spending which I think is accurate. I'm not sure that we're going to be able to cash all those checks. Like in other words, literally you all have so many money you can't spend it all as fast as you want. I think that's going to extend for 3, four, 5 years.

Wow. And how do you deal with the depreciation cycles that are involved there? Is the demand curve and the depreciation cycle curves match up?

Well, fortunately we buy just in time. But the nice thing is just in time for the hardware the depreciation cycle for the space power is more like uh somewhere between 25 and 40 years. So we have uh benefits there.

I think if you think of on the networking side and you look at both um um enterprise and the hyperscalers as well as neoclouds I think the story is quite different. So the the the enterprise is pretty nent in its buildout of true infrastructure. Um I just don't think that the data centers like if you assume that 100% of the data centers are at some point in time you will need to get rerracked and you will need a very different level of power um requirement per rack that's going to be there compared to what used to be there in the traditional data centers. I just don't think that um the enterprises are far enough along. Maybe the few enterprises that are at super high scale might be there, but I don't think the enterprises are far enough along.

Hyperscalers and NeoClouds is a completely different story. And uh to A mean's point on this notion of scarcity of power, compute and network being the three big kind of constraints in this thing. Um I I would say right now that because there's not enough power singularly in one location, data centers are being built where the power is available rather than power being brought to where the data centers are.

Um and that's why you're seeing a lot of projects that are being built out all throughout the world. The other point though is the um the the lion share of the constraints that we're going to have I I think are going to be sustainable for a for a long period of time. And as you have data centers that are being built farther and farther apart one there's going to be a huge demand for scale up networking so that you can have a rack that gets more and more networking for scale up. The second is you're going to have a lot of demand for scale out where you have multiple racks and clusters that need to get connected together. But we just launched a a new piece of silicon as well as a new chip and a system for scale across networking where you might have two data centers that act as a logical data center that could be up to 8 900 kilometers apart. Um and and you will see that just because there's not going to be enough concentration of power in a single location. So you'll just have to have different architectures that get built out.

Actually that brings us uh to the next topic that I want to discuss the future of systems and networking and so on and so forth. So Google brought about the first or at least large scale scale out commodity servers and production for the web revolution and now Nvidia is bringing back the mainframe in a different form. So what do you think happens next? I mean, is is this the new style of coherent clusterwide computing that we need and there's going to be shared memory and all sorts of things or do you think the pattern changes again?

I I don't think we're quite to um back to mainframes in that it is still the case that people are running on uh scale out architectures across these pools. In other words, whether you have GPUs or TPUs, you're not necessarily saying, "Hey, that's my GPU supercomput." You're saying, "I've got 16,384 GPUs." Y

and maybe I'm going to go grab some subset. Now I've got uniform all connectivity uh in many cases which is fantastic. Same with TPUs. It's not like I say I have a 9,000 chip pod and I have to make my job fit on that. Maybe I actually only need 256. Maybe I need 100,000. So I do think that actually the uh software scale out is uh still going to be there. Uh I'll note two things though. one you're absolutely right that say about 25 years ago uh at Google and other places simultaneously there was really a transformation of computing infrastructure like the notion that actually you would scale out on commodity PCs essentially the same ones that you could buy off the shelf running a Linux stack and that's what you would do for disk that's what you would do for compute that's what you do for networking I mean you all take it for granted that this is sort of it was radical there are many people who thought that this was a terrible idea that wasn't going to work.

I think the exciting thing about this moment right now is actually that we're going to be reinventing I'm not saying Google we are going to be reinventing computing and 5 years from now whatever the computing stack is from the hardware to the software right is going to be unrecognizable and by the way there was this code design because if you think about it I'll use Google examples because I know those best big table spanner GFS Borg Colossus they were handinhand codeesigned with the hardware the cluster scale out architecture picture and it was really the com we wouldn't have done the scale out hardware if you didn't have the scale out software same thing is going to happen in this moment so I I think actually the mainframe um it's going to look very very different.

okay yeah I do think there'll be like this ex uh extreme demand for an integrated system because like right right now we are very fortunate at Cisco where we do everything from the um from the physics to the semantics you know you think about the silicon to the application Um and the other than power, one of the constraints is how well integrated are these systems and do they actually work with the least amount of um lossiness uh across the entire stack. And so that that level of tight integration is going to be super important. And what that means the industry will have to evolve into is we will have to work like one company even though we might actually be multiple companies that actually do these pieces. And so when we work with hyperscalers like Google or others um there's a deep design partnership that actually you know goes on for months and months together uh ahead of the time before we actually even do them uh deal and then once a deal is done of course there's a tremendous amount of pressure to make sure that the you moving pretty fast but I think the industry's muscle of making sure that you operate in an open ecosystem and not be a walled garden is going to get important at every layer of the stack. Y completely agree.

And so let's talk about the disagregate the stack a little bit. One of the most interesting topic is processors, right? Clearly there's an amazing vendor producing an amazing processor that has massive market share today, right? And we see startups all the time doing all sorts of processor architectures. You got an amazing processor inside um your fortress. What do you think happens next in processor land?

Yeah, we're uh huge fans of Nvidia. Uh we we sell a lot of uh Nvidia uh products and chips. Uh customers love them. Uh we're also huge fans of our uh TPUs. Uh I think the future is actually really exciting and actually uh we're it's not that I don't think that we've hit the point of okay there's TPUs, there's GPUs, there's whatever tranniums or or something else. We're really seeing the golden age of specialization. And that that's my observation. In other words, if you look at it, a TPU, I'll use that example again because I know it best for certain computation is somewhere between 10 and 100 times more efficient per watt. And it's this watt that really matters than a CPU.

That's hard to walk away from, right? 10 to 100x. And yet we know that there are other computations that if you built even more specialized systems for, but not just a niche computation, computations that we run a lot of at Google, right? for example, uh maybe for serving maybe for agentic workloads that would benefit from an even more specialized architecture. So I think that actually one bottleneck is how hard is it and how long does it take to turn around a specialized architecture. Right now it's forever.

Yeah. Right. For the best teams in the world really from concept to in live in production speed of light is two and a half years. Yeah. I mean that's that's if you nail everything, right? And there are a few teams that do, but how do you predict the future two and a half years out for building specialized hardware? So, A, I think we have to shrink that cycle.

But then B, at some point when things slow down a little bit, and they will, I think we're going to have to build more specialized architectures because the power savings, the cost savings, the space savings are just too dramatic to ignore.

And this will actually have a really interesting implication on geopolitical structures as well. Because if you think about what's happening in China, China actually doesn't make 2 nanometer chips. They make you know 7 nanometer chips. Um and and so if you think about what but they have unlimited amount of power um and they have unlimited amount of engineering resource and so what they can do is do the optimization on the engineering side keep the seven nanometer chips and make sure that they give people unlimited amount of power. We might have a different architectural design where you have to get extremely power efficient. you don't have as many engineers as you might enjoy in China and you can actually go to two nanometer chips but and those might be power efficient in some ways but they might have thermal lossiness in other ways.

like there's a whole bunch of things that have to get factored in um on the architecture that'll get more specialized even by geo and by region and then depending on how the regulatory frameworks evolve uh you know how that that geo and expands like if China expands to different regions in the world you will have a very different architecture that plays out than if America expands to different regions in the world. So this is a very interesting kind of game theory exercise to go through on what happens in the next 3 years in in tech in general and no one knows right now. Yeah, that's the beauty of the world that we live in. Yeah. Yeah.

So we'll soon be measuring systems by engineers per token in addition to watts per token. Um all right so let's jump to another topic which very much engineer per kilowatt engineer per kilowatt in the US um networking right obviously you alluded to it um scale up scale out in your case you mentioned scale across so it seems to me that networking is also going to get reinvented in a fairly significant way so what are the leading sides that you're seeing that and the signals that you're seeing and on the direction networking is going to

Yeah, networking is going to need a transformation uh for certain. In other words, uh it the amount of bandwidth that's needed at scale within a building is just astounding. I mean and uh and it's it's going up. The network is becoming a primary bottleneck. Uh which is uh scary. So more bandwidth translates directly to more performance. And then given that the network winds up actually being a small power consumer that delivered utility you get per watt like it's a super linear benefit like spend a little bit here get way more there. So I think that uh that side is absolutely there.

Um I'll put in a plug here in that in this for these workloads we actually know what the network communication patterns are a priority. So I think this is a massive opportunity. In other words, do you then need uh the full power of a packet switch when actually you know what the rough circuits are going to be? And I'm not saying you need to build a circuit switch, but there is an optimization opportunity. The other aspect of this here is these workloads are just incredibly bursty.

Yeah. and and we're to the point where uh and we've written about this uh power utilities notice when we're doing network communication relative to computation at the scale of tens and hundreds of megawatts, right? Like massive demand for power, stop all of a sudden and do some network communication and then burst back to computing. So, how do you build a network that needs to go at 100% for a really short amount of time and then go idle?

Yeah. And then same actually for the scale across use case which uh we're absolutely seeing you don't run large scale pre-training across all your wide area data center sites 12 months of the year. So and then you're going to this is a problem I think about a lot is let's say you build the latest greatest chips in these three data center sites. How long are you going to be there before you migrate to the latest latest chips in three other sites? And then what do you do with the network that you left behind? People are going to run jobs on them. Yeah,

but you're not going to need nearly the network capacity that you did for large scale training, pre-training anyway. So the shift of needing massive networks for like 5% of the time it I I don't know how to build a network like that. So if any of you do, please um please please let me know. I mean, if you don't know how to build this, there's nobody that knows how to build this. We're trying to figure it out. It actually is a fascinating problem. Yeah.

Yeah. Right. I do think like if if you think of if power is the constraint and if compute is the asset I think network is going to be the force multiplier because you know if a if a packet if if you have low latency and low performance and high energy and efficiency then the packet the every kilowatt of power you save moving the packet is a kilowatt of power you can give to the GPU. Y um which is you know super important.

Um the the other thing is you know when you think about um scale up versus scale out versus scale up across you'll also need especially on inference versus training there are different things that get optimized like you might optimize for latency much more on training runs you might optimize much more for memory on inferencing um there there's uh there's architectural and so I I also feel like the way that networking will evolve is rather than it being um a training infrastructure that then gets applied to inferencing. You might have inferencing native infrastructure that gets built um over time. And so there's there's good considerations to look at on like how all of the architectural components are um are moving. But um in my mind like if if I were to say strategically one of the biggest things that's happening in networking from our vantage point is if you're just a wrapper around broadcom then you've got a monopoly that's going to be a very predatory one.

Um and so one of the big reasons where Cisco is um super relevant is you don't just have a broadcom world with people just wrapping mean that their systems aroundcom but you will actually have a choice of silicon and that choice and diversity of silicon is going to be super important uh especially for high volume you know kind of consumption patterns.

So last question on the system since you brought that up and we'll move to use cases. Um inference both of you have mentioned I mean you talked about in the context of the processors you just started talking about the architecture are you deploying today's specific architectures for inference I mean or is it still shared workloads? We are deploying uh specialized architectures for inference and I think as much software as hardware but the hardware is also uh deployed in different configurations is the way I would say it.

And then the other aspect of inference that is becoming really interesting is uh reinforcement learning uh especially on the critical path of serving because latency just becomes absolutely critical. uh and I think that so how you would build your system and how you would connect it up to one another and of course networking plays a a key role there uh becomes increasingly interesting.

and are there singular choke points that if removed would accelerate the thousandfold reduction in the cost of inference that we need or is this just a natural curve that we are writing down so so we're massive I mean two things here one again maybe many of you are familiar with this prefill and decode on inference look very very different. So actually ideally uh if you've uh you would have different hardware actually the balance points are different. So that's that's one opportunity. It comes with downsides. Uh we can talk about that.

Uh what I would say though is that maybe something people don't realize is that we're actually driving massive reductions in the cost of inference. I mean 10 x's and 100 x's. The problem or opportunity is the community, the user base keeps demanding higher quality. Mhm. not better efficiency. So just as soon as we deliver um all the efficiency improvements we're looking for, the next generation model comes out and it is the whatever um intelligence per dollar is way better, but you still pay more and it costs more relative to the previous generation and then we repeat the cycle. And it's almost like the longer um the reasoning that you have the more impatient the market gets, right?

Uh what I would say though is that maybe something people don't realize is that we're actually driving massive reductions in the cost of inference. I mean 10 x's and 100 x's. The problem or opportunity is the community, the user base keeps demanding higher quality. Mhm. not better efficiency. So just as soon as we deliver um all the efficiency improvements we're looking for, the next generation model comes out and it is the whatever um intelligence per dollar is way better, but you still pay more and it costs more relative to the previous generation and then we repeat the cycle. And it's almost like the longer um the reasoning that you have the more impatient the market gets, right?

So for example, if you have a 20 minute reasoning cycle like for example with deep research you could have autonomous execution for about 20 minutes that was interesting. Now you have you know most of the coding tools that can go up to 7 hours to 30 hours of you know duration of autonomous execution. When that happens there's actually a greater demand for saying compress that time down. Um, and so you'll it's it's kind of a self-fulfilling prophecy where you need to have more performance because of the fact that you've been able to go out and do things for a longer autonomous amount of time. And so it's almost a never-ending loop where you you'll need to have more performance for inference. Yeah. In perpetuity. Yeah.

Though intelligence per dollar is a business model metrics metric. So it is not just the processor capability. No, it's end to end. Absolutely. Yeah.

So okay. So let's uh change topics and talk about actual usage, right? So both of you have massive organizations. Where are the key wins that you're getting today? With with applying all the AI that's available to you and uh then we'll talk about what your customers are doing. But I'm actually curious about what you're doing internally wi within the teams. Yeah.

Yeah. So, so I mean coding is the obvious one and that's actually picking up uh increasing traction and incre increasing capability. Uh we just actually in the last couple of days uh published a paper that showed how we applied AI techniques to uh do instruction set migration. So in other words, we actually had a fairly massive migration from x86 to ARM making our uh entire codebase and at Google it's a very very large codebase uh uh sort of instruction set agnostic and including to you know future risk 5 or whatever else might come along uh tens and thousands hundreds of thousands of individual your entire codebase you're going to make it agnostic entire codebase because we we um want and need all of our codebase to be man that's a crazy ass project. Yeah.

So, so we we it it was and the the motivation though for this actually was a few years ago. Uh we had this uh amazing uh legacy system called Bigtable and then a new amazing system called Spanner. And we decided to tell the company, hey, everyone needs to move from Bigtable to Spanner. And by the way, Bigtable was amazing for its time, but Spanner was better. The estimate for doing that migration for Google was seven staff millennia. How much? How much? seven staff millennium that we we had a new unit that we had to actually to see what and and it was it wasn't like made up people being lazy. It's like this is this is what it was. It's endearing that they came up with that though and you know what we decided long live big table what it just wasn't worth it. Yeah.

Honestly like the opportunity cost was uh too high. So the and we have these sorts of migrations. So tensor uh flow to jacks we actually I mean again somewhat private but not not too secret we've affected this internally with AI assist went integer factors faster. Now there are other tasks which um the tools probably aren't quite yet up to the um whatever standard for but the the area under the curve is getting bigger and bigger and bigger.

So we're seeing probably like three or four really good use cases and then we are seeing some use cases which are not working yet. And so what is working code migrations is working relatively well so far we use largely a combination of codeex cloud and um and cursor some win surf and so um code migrations tends to work pretty well. Um, debugging, oddly enough, has actually been very very productive with um with these tools, especially with CLIs. Um, the um um where we've not done as good a job. And then front end 0ero to1 projects tend to do extremely well like the engineers are super productive. when you go to code that's older um and especially further down in the infrastructure stack much harder to go out and get that to happen.

But the challenge that we have to orient our engineers on this is actually much more of a cultural reset problem than it is a um just a technical problem which is if someone uses something and says this isn't working right um you can't put it back on the shelf saying this doesn't work for another 6 or 9 months. you have to come back to it within four weeks and see if it works again because the speed at which these tools are kind of advancing is so fast that you almost have to kind of get like so I was with a 150 of our distinguished engineers today and what I had to urge them to do is um assume that these tools are going to get infinitely better within 6 months.

Yeah. and make sure that you get your mental model to where that tool is going to be in six months and what are you going to do to be bestin-class in six months rather than assessing it for where it is today and then putting it aside for 6 months assuming that that's not going to work for the next 6 months. I think that's a big strategic error. So we've got 25,000 engineers. I'm hoping that we can get at least um two or 3x productivity within a very short amount of time within the next year. um and we it'll we'll be able to see what if um if that happens.

The second a couple of the big areas that we are starting to see some good responses is in sales preparation going into an account call really good legal contract reviews actually much better than what we had thought. Um and then the last one is not super high inference volume but product marketing. Um, I think the first chat GPT take on competitive is always better than what my any product marketing person comes up by themselves. So, we should never start from blank slate. Just start from chat GPT and then go from there.

Okay. Now, we could be talking about the topic for a long time, but they showed me the two-minute warning. So, I want to focus on one last question here. So, we got a lot of founders here, right? Building amazing companies. So what is the most interesting development they should look forward to in the next calendar year let's call it or the next 12 months a from your company and B from the industry if you are look at your crystal ball I mean I think to build on the point uh these these models are getting more spectacular uh by the by the month and then they'll be from whatever companies you like uh a bunch of really excit including ours I forgot to say you're not allowed to say models will get better. Yeah, everybody knows the models the models are going to get but I mean they're getting scary good is the part that I would say um but I think that then the agents that get built on top of them and the frameworks for making that happen are also getting scary good. So the ability to um have things go quite right for quite long over the coming 12 months is going to be transformative. on anything. Do you want to leak any aspect of your road map next 12 months?

I mean I think to build on the point uh these these models are getting more spectacular uh by the by the month and then they'll be from whatever companies you like uh a bunch of really excit including ours I forgot to say you're not allowed to say models will get better. Yeah, everybody knows the models the models are going to get but I mean they're getting scary good is the part that I would say um but I think that then the agents that get built on top of them and the frameworks for making that happen are also getting scary good. So the ability to um have things go quite right for quite long over the coming 12 months is going to be transformative. on anything. Do you want to leak any aspect of your road map next 12 months?

Not so not right now. Yeah. Okay. Do you

I I I'd say the the big shift and what I would urge startups to do is don't build thin wrappers around models that are other people's models. I think the the the combination of a model working very closely with the product and the model getting better as there's feedback in the product is going to be super important. So you are going to need foundation models but if you just have a thin wrapper I think the durability of your business will be very very um you know shortlived. So that would be something that I would I would urge you on and I think the intelligent routing layer of some sort that says I'm going to use my models for these things. I'm going to probably use foundation models for other things and dynamically keep optimizing will be uh I think cursor does that pretty well. Um but that that'll be a a a good way that the the software development life cycle will evolve.

Um what you should expect from Cisco is look truth be told for the longest time people thought Cisco was a legacy company like that they were has been and I think in the past year hopefully you've you've paid attention I think there's a level of momentum in the business there's a spring in the step in the employee base so uh you should expect like I said from the physics to the semantics in every layer from silicon to the application a fair amount of innovation in uh silicon and networking and security and observability ility and the data platform uh as well as applications um you know from us and um we're excited to work with um the startup ecosystem and um so if you if you ever feel like you want to work with us make sure that you reach out to us.

I mean one aspect that I I want to highlight about the models is um where we were with let's say text models two and a half three years ago they were fun like hey write me a haiku about Martin did a great job. Now they're amazing. I think that what's going to happen in the next 12 months is the same thing is going to be happening with input and output of images and video to these models. And to the extent that even for images, imagine them as productivity and educational tools, not just okay, here's Martinez Superman on a like that's cool too, right? But using it for productivity gains and learning, I think is going to be really really transformative.

Awesome. So on that note, we'll have to end this session. Thanks for a great conversation. I mean, thanks Tito.