Speaker A: If you accept as the truth that we're going to be running at the limit, then what that means is that the way to get more intelligence is to be more efficient. We can't get more intelligence by applying more force if we're already at the limit. We have to be more thoughtful about how we use what we have. We build tools. We build external organs that help us solve problems. You know, we we have an external stomach. We call it a kitchen. Now, we're creating an external brain. What is the implications of an external brain? Pretty profound. Nobody actually really knows.

Matt Turk: Hi, I'm Matt Turk. Welcome back to the Mad Podcast. Open source AI is having yet another moment with powerful new models arriving almost weekly. And my guest today is one of the very best people to unpack it all. Brian Kentaro leads Neimotron, Nvidia's family of open foundation models. Now, not everyone realizes Nvidia has a massive effort to build frontier AI models, but it employs hundreds of AI researchers and Neotron 3 Ultra immediately became the number one US openweights model when it was released just a couple weeks ago. We begin this conversation with the state of open source AI and the race between the US and China and then we go deep inside Neotron for bit training hybrid member transform architecture mixture of experts multi-token prediction and multi-teer distillation all in plain language and finally we get a rare look at how a modern AI research organization actually runs how you get many brilliant minds to build one model instead of a 100 papers please enjoy this awesome conversation with Brian Katansarao

Matt Turk: All right, Brian, excited to do this. It seems that open source is having a banner year. So, you guys at NVIDIA just released Neotron 3 Ultra, which is an important moment and the best open-source openweight model in the US. That was just a few days ago. And then uh even more recently, GLM 5.2 came out and that was another moment. So it seems that things are accelerating in open source AI. It feels like a great place to start. What's your assessment about where we are and how wide the gap between closed source and open source currently is?

Brian Katansarao: Well, it's really exciting to see all of the energy going into open technologies for AI because we know that um open technologies make it possible for people to innovate. You know, the internet is such a great example of that. Um, we actually did have closed internets. I don't know if you remember things like America Online and Prodigy back in the day. Um, and they were great. Um, and open internet has also u been amazing, right? Like so many different companies have been able to figure out how to transform their work um, thanks to uh, an open technology. The application of the internet to retail is very different from the application of the internet to healthcare or manufacturing. But all of them have been totally transformed um by the internet. Um AI uh I believe is uh also a very transformational technology and also a technology that needs to be applied in very diverse ways and because of that I believe that open technologies for AI are really fundamental. Um and it's very exciting to see continued um investment and development of open technologies from uh for AI from so many different organizations around the world. Um uh and uh you know I I hope that that continues.

Matt Turk: And what's your sense for how far behind opensource is compared to closed sources? It's been the the big trend of the last few years has been this sort of narrowing gap. Do do you think that open source is almost there or the bar keeps getting raised by the closed source models?

Brian Katansarao: Well, I I feel like this question um uh it's maybe a tempting question because, you know, it's fun to set up kind of competition, but but I actually feel like the whole AI community is moving very fast. Um and if you look, for example, at the progress in AI, whether it's closed or open just over the past 3 months, it's been incredible. Um, and so if you're in a field that's moving really, really fast, I think that's more important than any particular gaps that might exist between different models because the most important thing is, you know, how is AI developing as a field?

Matt Turk: What do you think the drivers are to continue progress in open-source uh AI? Is that the community? Is that big companies like Nvidia being behind it? Is that the global competition with China? what propels open source AI forward?

Brian Katansarao: You know, I think there's a number of things that that are pushing open technologies for AI forward. One is just the demand. You know, there's so many organizations that want to customize AI and want to integrate it deeply into their work in a way that really requires open technologies for AI. And so, so I think um the demand is certainly there. I think also it's just um uh the best way to develop technology. Um, and we've seen this, you know, for for many decades that technologies developed in the open move quicker because we can all learn from each other. And um, in an era where we're undergoing the most exciting thing to happen in technology in our lifetimes with the development and the deployment of AI, um, what else do do computer scientists want to work on other than making AI awesome? And if working together as a community is the best way to do that, then that's also a driver that pushes the community towards openly developing technology.

Matt Turk: to ask maybe a slightly cynical question. there is at least a a a part of the community that's wondering whether open-source as an ecosystem not not Nvidia but in general has been progressing in part based on the ability to distill closed source models and in a world where we seeing the anthropics and fable fives of of the world starting to discourage distillation do you think there is a chance that opensource AI progress may slow down in that context or as a result.

Brian Katansarao: You know, in my mind, there's no question that when the um technology community decides to make huge investments in the most transformational technology of our time, that there's going to be rapid progress. Um and also that that technology is not going to be controlled by a small group of people. Um because that's just not the way that um the industry works. you know, we we um do our best work. Um uh we have the most impact with our work when we're able to uh each think about it in our own way and apply it in our own way. So um you know uh I love uh the uh closed AI APIs uh whether from anthropic or other people. I think they're amazing you know um really really impressed with the work that those labs are doing. But they're not the only labs in the world. There's lots of labs around the world and lots of people have a good idea. Um, it's not the case that there's only a few labs that have the monopoly on all good ideas. That's just not true. That's not how humanity operates. There's a there's a lot of bright people on this planet. And um, you know, the community uh, of course cares deeply about this technology. It's obviously so transformational, has such profound impacts on so many things um that that of course uh, many people uh, want to be involved in that. And um so I think over time we're going to see that um community oriented approaches to developing and deploying AI are going to continue to strengthen and be widely adopted because that's really the history of how we built things as a as a human uh species.

Matt Turk: Do you think that is globally true as well? So um you know in particular with respect to China this perception that yes a lot of people have great ideas around the world. Uh however a lot of progress from Chinese models were directly inspired or perhaps uh generated through distillation from from the closed source models. Is that just a kind of like press rage bait or from the perspective of a leading AI researcher? You're very impressed by the the the novel ideas that come out of China as well.

Brian Katansarao: You know, um uh perhaps unusually, I uh actually did work at a Chinese company for about two and a half years. I worked at BYU. Um I worked in the Silicon Valley AI lab uh along with Andrew Ing and as well as Daario Amadei and um uh we all worked uh for a Chinese company and saw how smart, hardworking, creative, inventive our colleagues were uh at the rest of BYU and you

Speaker A: ...know that experience has stuck with me. Um I think it's absolutely false to say that um you know uh the achievements of some other country are all being um created by sort of you know copycat mentality. It's just not true. Um now do we all learn from each other uh in the technology community? Of course you know of course of course we learn from each other. But um you know uh I would say uh you know it's been a really good thing for the world that the Chinese uh AI community has been so um open with what they've been building. I think it's enabled a tremendous number of companies to build things that uh they couldn't have done without um that community and I think it's also spurred um technological progress throughout the AI uh ecosystem. So, you know, I'm really grateful for um the contributions that um our um colleagues in China have made over the years and um you know, I would love to encourage a spirit of openness amongst AI labs around the world outside of China as well. You know, I was really excited when uh OpenAI released the GPT OSS models um a while back and then of course Google's been doing great work with Gemma. Absolutely thrilling to see that. Um, and you know, we're pushing Neotron along here at Nvidia as well. Um, so I think there's a chance for um uh the rest of the world to catch up to China uh in the sense that um you know we can understand the benefits of working together uh as a community to build technologies for AI uh in a way that I think China has frankly been leading.

Speaker B: Great. What is the case for a customer to be using open-source models uh these days? What is your fundamental advantage?

Speaker A: Every company is built around a secret. Uh this is a secret that has to do with not just their intellectual property but also their platform. Uh which has to do with how do they interact with problems and customers? um how do they think about solutions uh to what their customers need and it is always the case that the value of AI is greater when it can be more tightly connected with those secrets because you know AI depends on data critically so the more valuable the data that goes in the more valuable the solution becomes now um every company when it's thinking about how to deploy AI has to think through what are the implications for the core secrets of our company and um there's a lot of circumstances where um due to trade secrets or or you know trying to think of think through the business model or or even regulatory requirements that you know there's data that you really have to treat very carefully by law. Um and it is much better to do that when you are able to think that through and implement it yourself. um thinking about the integration of AI, the way that AI interacts with um customers, um the guard rails that are put in place, you know, every company um has a specific understanding of its customers and and therefore what the customer needs. And um the amazing thing about open technologies for AI is that they allow custom uh customization, right? So companies can think this through. They can build things that really matter for them. And you know, I started out this conversation talking about the internet and about how the internet, the deployment of the internet has been done in very different ways for very different industries. Um, and there's a lot of desire to do that. Um, uh, as we see AI change the way that we work and play throughout the entire economy. Um, this is really spurring a lot of demand for open technologies for AI.

Speaker B: Great. I'd love to go into a bit of a deep dive into Neotron, but before we do that, maybe a few minutes on your story, your background. What was your path to where you are today? Um, including the the BYU detour.

Speaker A: So, um, I started work at, uh, Nvidia in 2008. Uh, at the time I was a graduate student trying to figure out, um, parallel computing for artificial intelligence. And I thought um Nvidia had a chance of changing the way computers work AI.

Speaker B: which was a lonely presumably a lonely quest right in 2008.

Speaker A: Oh, it was it was very chaotic back then. People thought I was crazy and you know I remember going to ICML in 2008. I published my first paper training models on the GPU and people asked me why I was there. People said this is not a good paper for ICML. We're we're we just do fancy math here. And I was like, well, but I think computing actually matters a lot for AI. If we could train bigger models that had more capacity to learn, we could probably solve more problems. And um they kind of nodded their heads and like, well, I'm not really sure why you're here.

Speaker B: Isn't a GPU a thing for gaming as well, presumably, right?

Speaker A: Yeah. There's there's also that, right, which we we continue to um run into that uh that idea. Actually, a GPU is whatever Nvidia says it is. You know, we make them. So a GPU is is a is a thing that we make in order to accelerate the world's most important computations. Um which in 1995 was graphics and you know for a long time now it's been AI. So um anyway I started at NVIDIA um I was in the research group um doing uh strange things about trying to make um uh compilers libraries uh for AI on the GPU. Um that led to um the creation of uh first copperhead which was a it was a a Python uh embedded language that um compiled to the GPU um which I think foreshadowed a lot of things in TensorFlow and and PyTorch um and then um uh then that led to the creation of QDNN which was Nvidia's first product for um uh for deep learning on the GPU. Um and uh I I really enjoyed working on that. Um but I was always wanting to see more uh firsthand about the applications of AI and at NVIDIA I was mostly working on you know uh libraries and compilers for AI. So I thought um well you know when Andrew Ing asked me to go uh build the Silicon Valley AI lab with him uh at BYU I thought oh this is a great opportunity because even back then BU um was very advanced in its application of AI to its core business. And so um uh so that was a a fantastic opportunity for me. The BU Silicon Valley AI lab was an amazing place um full of brilliant people that were working really hard.

Speaker B: What was it like working with a young Daario? Uh was there like any signs that he could become uh you know who who he has become?

Speaker A: Daario um was brilliant from the beginning. I remember um I interviewed him uh I was on the panel um and um at the time he uh had been working in bioinformatics so he he hadn't been working on deep learning or or the things that we call AI these days um but it was very clear that he learned extremely quickly and also that he thought extremely deeply um I think you know uh the thing I admire most about Daario uh is the strength of his conviction um you know I've been working in this field uh for a long time and I've believed also that AI is going to transform the world but I don't think that I believed uh in it as completely as Daario did and perhaps that was because you know my academic training during my PhD was full of a lot of caution uh I don't know if you remember but AI was old and bad in 2005

Speaker B: it will never work

Speaker A: that people did with computers they started doing it in 1945 Right. Um and and so there had been so many grandiose promises that failed to deliver over the years. And so I came to AI with a lot of caution. In fact, back then we used to call it machine learning, which was basically a dodge. Like we just didn't want people to know that we were we were working on AI because then they would be like, "Oh, we've heard about that. It never works." Right? So, um I came to AI with a little bit of this like uh you know, academic caution like, "Oh, we should you know, we should hedge a little bit. like I don't know if now's the time like um and and Daario, you know, he his uh strength of conviction and his understanding of the moment of how um the technology was developing this time it was actually going to work and then the implications of that on um you know how the technology should be developed um uh what kind of institutions uh to build I think he's done a spectacular job and so um yeah working with

Speaker B: ... it was always always a a fun experience.

Speaker A: So then you went back to Nvidia and walk us through the journey.

Speaker B: Yeah. So uh 10 years ago actually in 2016 uh Jensen called me up and said hey uh would you like to come back and build an applied research lab and I thought that would be a fantastic uh opportunity. I you know I've always loved Nvidia. I've loved um the way the company works, the convictions the company holds. You know um Nvidia is a very unique company. it follows through over long time periods, you know, uh, and I've seen that with CUDA. I've seen it with our deep learning technologies. I've seen it with our ray tracing graphics technologies, our AI for graphics, you know, over and over again, Nvidia is not afraid to put in five or 10 years worth of research in order to change the world, you know, and um, working at a company that has that strength of conviction and the ability to follow through is kind of an ideal thing for me. Um I just really I just really love the support that the company gives uh gives its researchers um to invent the future.

Speaker B: And so um I thought I'd come back um uh the first project that uh that I worked on um actually became DLSS uh which uh some of your um audience may know about but DLSS is our real-time uh AI for graphics and it makes a small GPU run like a big GPU. It's about 10 times more efficient because rather than computing uh the color of every pixel for every frame, we use AI to infer the color. Uh and uh you know these days 23 out of every 24 pixels uh is being generated by our AI model when you're using DLSS um to play games. And gamers love it. It's become the standard way of playing games because it's just so much more responsive and it's more beautiful. Our AI, we train it offline on huge data sets. Um and it's able to render uh graphics in real time uh more beautifully than uh traditional methods do. Um uh we recently actually announced DLSS 5 which is a fully generative version of DLSS and um I am so excited about it. It represents culmination of 10 years worth of research on how to make um real-time graphics much more beautiful. Um and so uh so that's part of uh part of the journey uh here for me was real-time uh uh AI for graphics.

Speaker B: Um but then at the same time we also started a language modeling project. Um and this was back in 2017 um you know before transformers uh were big and before um language modeling uh uh started taking over the world. But um you know I just had this intuition maybe built on you know some of the things that that I had seen uh while working uh at BYU. I I just had this intuition that you know working with text and understanding text was going to lead to better reasoning which was going to lead to better application of AI in all sorts of domains.

Speaker B: And so um uh so we started this project uh called Megatron. Megatron stands for the biggest baddest transformer. That's why we named it that. Um, and it was really a systems project to show the world how to train the largest transformer models on Nvidia's hardware. Uh, back at the time, uh, some of your, uh, audience may or may not remember this, but, uh, there was there were being claims made that the only way to train big transformer models was on the TPU. Um, because after all, the transformer had been invented at Google. And, uh, so, you know, we looked we looked at uh, you know, we loved the the the transformer paper. We thought, wow, this has amazing potential. We tried it out on our own language modeling tasks and it worked so much better than the RNN's that we had been using before. And also we saw immediately that there was an enormous systems opportunity to co-optimize the GPU, the networking, all of the compilers and software that would enable people to scale uh transformer-based language models uh really dramatically. And we we thought, you know, this is this is something that that um could really have an impact. So we started the Megatron project um which then led to I think uh basically helping the whole industry figure out how to train um extremely large uh LLMs um and also led to the foundations of today's Neotron project um where you know Nvidia trains uh its own LLMs um uh for its own purposes. So that's kind of the the history

Speaker A: great journey. Okay, so let's go into all things uh Neatron. And before we get into the specifics, that's the obvious question that I'm sure you've been asked uh many times uh which is why does Nvidia care in the first place to be building model and investing very significant efforts into creating its own family of of frontier models.

Speaker B: You know, Neimotron has two jobs. The first job is to help us understand how to build the systems of the future. NVIDIA is an accelerated computing company and that means thinking through the world's most important computational challenges from first principles and designing systems which includes a lot of software uh in order to make it possible for people to invent and deploy things that never could have been done with standard computing. But in order to do that, Nvidia has to deeply understand everything about how AI works. That's how we co-design all of the systems and software um uh for our main product line. So the first job of Neotron is to make sure that Nvidia continues to exist so that we can continue delivering meaningful acceleration in an era where Moore's law has died. uh and the the acceleration that we get these days comes through specialization but again specialization comes through understanding.

Speaker B: So that's Neatron's first job is to help Nvidia understand how to build its core products. Nvidia's second or Neatron's second job is to support the ecosystem. Uh, one of the most valuable things that Nvidia has built over the years is um, all of the people around the world who build and deploy amazing AI um, using Nvidia's uh, technologies and uh, we think that it's necessary for uh, open technology for AI to continue to exist uh, from Nvidia to help uh, support that. Neotron's not trying to be the only open technology for AI. We love all technology for AI for the very straightforward reason that whenever AI uh is further developed and further deployed, it's an opportunity for our business. So, so this is this is um you know, we're we're very explicitly trying to develop our ecosystem because that's good business for us. Um but we're not trying to be the only provider of technologies for this ecosystem. We love seeing um other companies contribute as well. Um the the most important thing for Neimatron's second job is just making sure that it continues to be possible for companies of all shapes and sizes to build and deploy their own AI.

Speaker A: By the way, Moors law is dead. Is that is that is that official?

Speaker B: It's been dead for years.

Speaker A: It's been dead for years. Like why why is that?

Speaker B: Well, you just look at the the progress uh in semiconductor manufacturing. You know, the the original statement of Moore's law was economic, right? It was about we can afford to put twice as many transistors on the same chip in every whatever 24 months, whatever the the time period is. And um these days that is absolutely not the case and it hasn't been for probably 5 or 10 years right now. We are still scaling our systems right um through a number of ways. One is just applying a lot more silicon to it, right? uh we are also getting transistors are continuing to get smaller and and more efficient although at a slower pace but they're also getting quite a bit more expensive at the same time. Um uh so the uh you know in an era where where Moore's law was alive, the best way to make the system of the future was to take the system of the present and then just shrink it and and maybe double it at the same time, right? But in an era where where we've been living for a while now where you don't get economic benefits from taking your existing design and shrinking it, uh you really have to be more clever about how you use every part of the system. uh that that's uh you know an era where accelerated computing is is much more valuable than ever because the the work of thinking through the pro problem from first principles and co-designing absolutely everything uh from transistors to algorithms and applications uh in order to reduce

Speaker B: waste and and deliver meaningful acceleration that's more valuable than ever.

Speaker A: Fantastic. to play back what you were saying uh a minute earlier. It makes good business sense for Nvidia to be in the model business because one it helps design better chips and two uh whatever is good for AI is ultimately good for Nvidia which makes a lot of sense. That Neotron effort is reasonably recent, right? It started in 2023, I believe. Maybe walk us quickly through the key releases. I believe in 2023 there was Neatron 38B as a key release or am I missing a step?

Speaker B: Yes. Yes. Yes. So, you know, the the you know, the the numbering is somewhat lost to time. It I almost feel like we're in the Lord of the Rings and it's like, you know, there's like some ancient like relics that we're digging up out of an old mine. Um, you know, this is a long time ago. You know, the original what what uh we originally called Neotron 1 was actually a project that we did with Microsoft. Um, we jointly trained a 530 billion parameter model. Um, I believe that was released in 2021. And so this is GPT3 era. Um, and that's what um, at at the time we called it Megatron Touring NLG. uh touring was uh what Microsoft was calling their their language model efforts um at the time. But uh uh that uh in retrospect we called Neotron 1. Uh then along the way we built a few more uh we got up to Neotron 3. Um and then uh Llama came along uh and we were really excited about that. We were very happy that Meta was supporting the open uh AI technology space. Um and so we we started um you know taking our language model technology and adding it to llama models which then resulted in llama neatron 1. Um and you know that was the first uh reasoning model uh built on llama. Uh we were really proud of that and

Speaker A: that was 2025.

Speaker B: Uh might have been 24 uh I I believe I can't remember uh somewhere around there. Um and then uh uh yes and then we we continued uh to to develop that um uh and uh you know last we we so we the numbers kind of started over again. We released a a Neotron 2 um I believe it was last year. Um and then we quickly followed that up with Neatron 3 because um uh we we needed to put support in. Neatron 2 didn't have support and that made it kind of uncompetitive against uh other models like GPT OSS 20B was just like so fast because of so we were like okay we've got to we've got to put put thee in. So that became Neatron 3. Um now we're in a a slightly difficult state because you know we're working on Neotron 4 right but we already released a Neotron 4 which was um in 2024 we released a 340B uh model called Neotron 4. Um, and so I'm not exactly sure how we're going to um solve this marketing problem. I didn't create this marketing problem. Uh, so uh I'll I'll I'll do my best to to make it clear that Neotron 4 of of of whatever the NE whenever we release that is different from the the 2024 Neotron 4. Um, but uh in any case, we've been working on this for a long time. I I think um more important to us than any particular generation is just the sustained commitment that Nvidia has to developing these models. We've been doing it for a while. I think our models have gotten dramatically more useful in the past year. Um which is a reflection of two things primarily. One is that uh the whole company has come together. So there are many different teams around Nvidia that now understand how important this is to Nvidia's future. And so there's dramatically more people and better ideas that are going into Neotron. Um and then uh number two along with that we've been able to scale the um compute resources that go into it. Um obviously it's um very important to have good computing infrastructure to build AI. We've um recently increased our investment substantially because uh we believe that that this is really really key to our company's future.

Speaker A: Fascinating.

Speaker B: But I just to continue the thought I think it's really important that everybody knows that we've been doing this for a long time. We are increasing our investments substantially and Nvidia is a company that follows through. You know we followed through over 10 plus years with CUDA and we're doing that with Neotron. Now

Speaker A: that's very helpful because uh I think the the broader world is is just starting to catch up to the fact that there is a very substantial open-source frontier AI research effort that that's been happening. So it's really interesting to to hear that uh you know there's been this progression and now there's this family of models that we're going to talk about uh in a second. Another important moment seems to be uh the creation just in March 3 months ago of the Neotron coalition. Do you want to explain briefly what that is?

Speaker B: So, Neotron exists to help support the ecosystem and we were thinking, well, this is a different kind of AI project than other projects around the industry, right? Because um we're not actually trying to dominate in any way. We're just trying to support we don't we're not trying to control uh uh the way that AI is um uh being integrated into all these companies. We're just trying to make sure there's good AI. But we thought well maybe if we worked with people while we develop it then it's going to be more useful for them. It'll be easier to integrate because we will consider what they need from the beginning. And um you know Neotron has always been collaborative. I was telling you that you know long long time ago our first big model that we trained we we did with Microsoft right? It was a joint effort where Nvidia and Microsoft researchers worked side by side to build that and that um that ended up I think helping both Nvidia and Microsoft. I think we both learned a lot from that um experience. And so because Neotron is not trying to compete with uh other companies, but rather support, uh because we're going to be putting it out there openly anyway, why not collaborate before the thing is built? Rather than Neotron being a project that Nvidia does all on its own and then posts on the internet and says, "Hey, why don't you try this? We think it might be good. Why don't we make sure that it's good for the partners that um uh that are interested by working with them before Neotatron is even created um and incorporating you know any sort of feedback evaluations environments um uh benchmarks um or any other um kinds of technology that other people want to bring. It turns out that the entire ecosystem there's a lot of companies that really want open models to succeed and so they have a self-interest. they have their own vested self-interest in making sure that open technologies are excellent and so why not um work with them and and let them contribute uh however they'd like to to making Neotron better. So that's the the idea of the Neotron coalition. It is not an exclusive coalition. We're not trying to be the only model out there. All the companies that we work with are free to to continue doing the work however makes sense to them. And yet um you know these companies want to work with us because they want to make sure that um open technologies for AI keep um developing quickly and that they have a chance to influence how that happens.

Speaker A: Great. What's the current state of the Neimontron family? You got nano, you got super, you got ultra. What do those models do and what are the use cases for them?

Speaker B: So nano is a 30 billion um uh total 3 billion active parameter model. supers 120 and 12 and ultra is 550 and 55. Um they're designed um really to fit, you know, it's kind of small, medium, and large um uh deployment scenarios. Um uh you know, Nano can be really capable for things that um you know don't require nearly as much knowledge or reasoning, but obviously for the for the most um capable model, you go for ultra. Um, super in a lot of ways is our most popular model because it represents kind of a great balance between um cost and and intelligence. So we we kind of like um having this small, medium, and large um approach to building a family just because our customers um seem to respond to that um pretty well. But um you know uh the most important thing from Nvidia's point of view that people are doing with uh uh with LLMs is agents right is um building agentic

workflows having an agent working on your behalf solving problems for you night and day. Um such an exciting way of approaching the problems that we have to solve. Um and um it's our dream to make Neotron amazing for that purpose. That's our goal

to double click on this at a high level. Neotron family is focused on agentic reasoning with a particular focus on making it efficient. Is that is that the right headline?

That's right. Yeah. Um Neotron has always been um uh speed first approach to building models because NVIDIA is an accelerated computing company. As I was saying, we're trying to think through what is the problem here computationally from first principles and um you know Neotron uh 3 family has a lot of things in it that are uh we're really proud of. For example, um Neotron Ultra and Super uh were pre-trained using 4bit arithmetic. We pre-trained those in MVFP4 um which you know uh is a not trivial thing to do to invent the algorithm so that your model can converge to an excellent result using such coarse arithmetic uh required a lot of invention. Really proud of that. Do you want to explain maybe for for people what 4bit is versus 16 bit for example?

You know, actually there was a fantastic post I saw in Hacker News yesterday where somebody let you um upload a picture and then it would basically posterize it, basically reduce the colors to fit different um uh number formats including NVFP4 and MXFP8 and some of the other formats that are out there. And so you could kind of swipe around and look what it does to the colors of a picture. Um and you know it's it's really quite dramatic. Four bits is not a lot of bits, right? That's only 16 values. Um now, of course, these are all um what are called block scaled formats. So, um groups of numbers also come with uh an 8bit um scaling factor. And the the specifics of this can get rather complicated. So, maybe they're not quite as important. Uh but the the reason why we want to do this is because first of all we have dramatically higher um throughput for these formats in our GPUs um specifically on Blackwell Ultra. Um and uh uh secondly, we know that it's going to save an enormous amount of energy. Um one one way to think about uh the computational problem of AI is that we are going to be running at the limit. Whatever the limit is, it could be uh an economic limit like we only have so many billion dollars to to buy servers with. It could be a power limit. We only have so many gigawatts that we can afford to to train a model with. Whatever the limit is, um uh we're going to be running at that limit. The or every organization is is because why? Because the value of intelligence is so high, you know, that that people are going to they're going to invest because they they know that they're going to get return. um uh the value of intelligence is is enormous. Um so if you if you accept as the truth that we're going to be running at the limit, then what that means is that the way to get more intelligence is to be more efficient. We can't get more intelligence by applying more force if we're already at the limit. We have to be more thoughtful about how we use what we have. And you know, 4-bit number formats are dramatically cheaper to move around. They take up less space in memory. um they take up less uh pico jewels when you move them from the memory um in or even on the chip uh around the chip much less uh energy when you compute on them and so uh so that's really um driving you know the investment in 4-bit formats and I think these days 4-bit formats for uh deployment are very well established um it's it's pretty pretty um straightforward these days to make a a good quantized uh four-bit uh checkpoint that you can deploy and that gets you a lot of inference cost and speed advantages. Um but using 4-bit formats for pre-training uh that's quite a bit uh more challenging because uh you have this um numeric uh solver that's you know optimizing the weights and um you know it it can be quite sensitive. So if you if you don't uh treat the numbers right uh your model can diverge uh and instead of actually getting a model done through pre-training you end up with you know basically just uh that run diverged which is you know always always scary. So it it took a lot of invention for us uh to be able to pre-train Neotron uh in 4bit. We're really proud of that.

Okay great. All right. So, as we uh get into slightly more technical things, the uh architecture of Neotron uh is hybrid. Is that is that right? So, it's a combination of transformer and member state space which is a slightly more exotic uh form of of of architecture. Walk us through that.

Yeah, you know, we published a paper in 2024 that showed that you actually get a smarter model by combining um state space models with transformers. And we we actually did a sweep of uh you know, how much of the model should be full attention and how much of it should be um a state space model in order to get the the lowest perplexity, basically the best the best language model um that you could get. And we found that you actually want it to be mostly a state-based model with a little bit of attention. And so kind of the intuition behind that is that um the state space models seem to be better at um kind of this intuit um intuitive kind of impressionistic understanding of a sequence um uh because they're um you know they're kind of summarizing the entire sequence into a constant space. That's how they work, right? So instead of having the ability to look at the entire sequence randomly, uh they summarize everything at every step into a constant um cache or little scratch pad that they're they're working on. And um that constraint seems to actually make them smarter at some tasks that involve like global understanding. Um on the other hand, the advantage of full attention is that it can pick out very specific uh bits of information and look at those exactly. It doesn't lose anything. There's no lossy compression going on. and you can actually see the whole thing. Um, and so we found that um, uh, you know, using both of these together was actually better than using either one on their own. Um, and that is independent of the speed benefit. That is just the model is smarter. And you know, since we published that, I think a lot of other um, uh, labs have also found this to be true. Um, you know, a lot of uh, uh, models these days are being built with hybrid SSM approaches. For example, QN has done that. Um, Kimmy is using what they call Kimmy linear attention these days. So, um, it's become, I think, quite widely adopted to use some sort of state-based model in conjunction with um, full attention for um, uh, for the the base architecture. Um now it also has some speed benefits because um the uh uh amount of memory that you need to hold uh that state space uh cache is actually constant with respect to your sequence length. um which then means that um generally you can fit much higher batches on the GPU when you're training and doing inference uh because the memory um requirement is lower and it keeps the GPU uh fuller and busier um and therefore um you know provides some some pretty important signific uh some some pretty important efficiency benefits as well.

So the models are also based on ane mixture of expert architecture. walk us through that and maybe remind people what is in the first place.

So mixture of experts is a form of sparsity. Um the idea is wow, you want to train a model on the entire internet. You want it to remember absolutely everything about the history of everything. But when you're answering a particular question, does it seem reasonable that it needs to actually think about the entire universe in order to answer that question? Actually, no. It seems like it's quite sparse, right? It seems like um we're we're we're using a language model to explore a very tiny space of ideas in order to answer a question or solve a problem. We want the model to be able to draw from the entire universe. We want to train it so that it understands everything that it possibly can. But when it's actually running, it doesn't really need to see all of that information. There's been a variety of approaches to

Speaker A: that try to take advantage of this property, but mixture of experts has been the most successful. And the way that it works is that the neural network has what's called a router that is learned that um is going to decide to send activations to a subset of the experts uh for every token that's flowing through every layer of the model. It's going to be making choices about um which fraction of the model is going to actually get to interact with this token as we try to understand it, build up representations of the problem, and then generate the next token that we're going to output. So it's a little bit like if I have a company with 550 employees but uh 55 of them are in engineering I want the 55 employees who are specialists to come to my meeting about engineering and not the rest of the company.

Speaker B: That's right. Yeah. Or you could think about it as a library. Like if you go into a library to do research, you don't read all of the books in the library. Like your first job is to figure out which books do you need to look at in order to find the answer to your question. And so um so that's kind of the the idea behind. Now have fascinating implications for the systems that we build. So with Blackwell for example, Nvidia went all in ones. That's why we built NVL72 which allows up to 72 of our GPUs to read and write each other's memory uh at very high speeds, very low latency. Now why is that important? It's because as you put a token through the stack of layers, at every layer, you have a router that's routing that token somewhere else. Why don't you partition your experts so that you know the experts are not sitting every expert on every GPU, but you have a subset of the experts assigned to each GPU and then you're routing the tokens between the GPUs very dynamically as you push uh the token through the network. Now um this is impossible to predict in advance where the tokens need to go because it's very specific to that particular token for that particular model. And so that's why we built NVL72 and that's why um Blackwell is so amazing for inference for you know today's AI models uh is because we thought deeply about mixture of experts when we were building it and this is speaking to Neatron's first job. you know, if if we hadn't been working on understanding AI, we wouldn't have been able to build Blackwell properly. And that, you know, has has translated directly into um you know, increased deployment um of Blackwell, which you know, we're we're we're very excited about.

Speaker A: Is what you just described called latente or is that a different concept?

Speaker B: Latent MOE is a specific uh innovation that we have in Neotron 3 family and uh what it does is actually um reduces the amount of communication that has to be sent through envy link during computations by basically down projecting it. So you know every token is it produces a vector and the idea is like we're going to take that vector and learn a way to compress it and then send that compressed thing through the network and then we're going to uncompress it at the other end and as a result we save on network bandwidth and we also get four times the number of experts for the same inference cost. So you could think about it as like you know our library of books got four times bigger um and we get to you know read four times more books uh at the same inference cost because of because of this particular innovation

Speaker A: is in general becoming the default architecture for Frontier AI.

Speaker B: Yeah I believe have been the default um in Frontier AI for a long time. Um they're just a really good combination of inference cost and intelligence.

Speaker A: Great. Great. But they have drawbacks as well. You know, they they take a lot more memory. Um uh if you have a very small amount of memory, a dense model is going to be smarter. Um and they also um they tend to work best either if you're running at batch size one. So you're running basically a single job or you're running a huge data center with like infinite queries coming in in the middle. They can be a little bit tricky.

Another important characteristic of Neimatron 3 Ultra is the 1 million token context. the the long context window. How important is that in the overall mix and what does it enable the model to do?

Speaker B: The longer the context length, the more challenging problems we can solve with a language model. Um, that allows us to do things like append all sorts of information to a query, which could be a codebase, it could be instructions. Um, you know, uh, in in the long term, I'm hoping that I have my own personal LLM that's able to read all of my emails, you know, and help me answer questions about that. You know, the more information that we can attach to a particular query, um, the the more useful the model can be. Um, now, uh, it can get more and more expensive, right, to reason over large amounts of of input data. And um and so that's one of the the reasons why there's usually a limit on how big the context length can be. But with Neotron 3, we we tried to push it as far as we could go. Uh we think a million tokens is a lot of tokens. Um and you can do a lot of things with that.

Speaker A: Prisma is particularly helpful in sort of multi-step agentic workflows and there's this whole separate discussion around context compaction to make sure that the model doesn't get lost in too many tokens. So like how do you how do you all think about this?

Speaker B: 100%. I mean um uh you know compaction is that's a thing if you're using an agentic workflow you deal with all the time. Um and compaction tends to work pretty well you know because language models um are pretty good at at identifying the most relevant things and summarizing and you're basically trying to summarize your context when you compact it. Um so uh compaction uh is is not a bad approach. I think having models that can just natively uh reason about larger amounts of data is just inherently more useful. So of course we want to push the the boundary on that as well.

Speaker A: Great. Can you talk about the multi-token prediction uh which is also very interesting

Speaker B: if you're running at a low batch size um which is when you are trying to get the most interactivity if you're in a data center. So you want you want the model to respond as quickly as possible and it's okay for it to be more expensive. your token your cost per token might might be higher but you want the result as quickly as possible or if you're running um locally um so you might be running at batch size one just because you're the only person using it. It turns out that uh the GPU has extra execution capabilities that are just lying there unused. The bulk of the work when you're running in these scenarios is actually fetching the weights from memory. And then you push the token past those weights and then you fetch more more uh weights from memory. But it turns out if you if you push two tokens or even five tokens through those same um weights, it would cost basically the same amount of time because the the expensive thing is not doing the math to push the token through the weights. The expensive thing is just reading all of those weights from memory. all those parameters they have to come in. And so the idea with multi-token prediction is to take advantage of this by having the model predict multiple tokens at once. Let's say that the model predicts five tokens. We know the first token is correct. The next four tokens may or may not be correct. So then what we do is on the next pass we take those four tokens and we stick them into the model uh and then uh run it through and at the end we check you know the model then predicts another set of tokens right then we check were the extra tokens we predict last time correct. If so then we just accept them uh and then we get like a 4x speed up. um and if they were incorrect then we only accept the ones that were correct and then um you know uh uh proceed from there. So the benefit of this is it it doesn't degrade accuracy at all because you're using the model to doublech check right. So all this speculation is going to get checked uh during the next token that you you uh run through the model. So it doesn't degrade your accuracy at all to turn on

Nvidia Representative: Multi-token prediction, but it can give you a speed up and it's probabilistic depending on the acceptance rate of your predictor, you know. So if your predictor is more accurate, the acceptance rate goes higher, you get a higher speed up. So with our recent Neotron models, we're pretty proud of our acceptance rates, but we're always trying to make them better. Always trying to improve that acceptance rate. This is a really good example of accelerated computing. With multi-token prediction, the speed that you get is a function of the accuracy of your model. The more accurate your model is, the faster the inference is, the cheaper the inference is, the more accurate it is. That's not usually how it works, but in this case, that's how it works. And what that implies is that if we're trying as Nvidia as a company to provide meaningful acceleration to the world's most important computational workloads this has to be an important part of how we think about it. If there's a 3x cost reduction or speed improvement for inference, which is the most important computational workload of 2026, if that's on the table and it depends on the accuracy of the multi-token prediction network, then that's something that Nvidia needs to understand very deeply because it's going to affect our business directly.

Interviewer: Fascinating. To continue on the tour, multi-teer distillation, we talked about distillation a little bit up front. What does that mean in the context of Neotron 3?

Nvidia Representative: So with Neotron 3 ultra we did postraining using something called multi-domain on policy distillation and what that entails is that we have many different aspects of the model we want to improve. For example science understanding is different from math theorem proving which is different from coding which is different from agent harness interactions right. With Neotron 3 I think we had about 10 or 15 of these teachers and so the idea is that you take these teacher models and you push them as far as you can go on some specific domain. So you just don't worry about making it good at everything just make it really really smart at this one domain. Then you have a collection of these models and you want to create one model that learns to be good at everything. And we do that using a specific reinforcement learning technique that a lot of labs these days use called MOPD. And the good thing about this is that because the teachers are supervising they can give really dense rewards to the student model. Basically every token is getting supervised and so the student can learn really quickly and then become almost as good as all of the teachers at all of the things. So one benefit of this is that it really helps the team work together better. If you don't have a technique like this and you have let's say 500 people working to try to make a model better and one team's like well I'm trying to make it better at this thing and then another team's like I'm trying to make it better at that thing there can be a tug-of-war where it's like well who wins. And if you have to make a choice like oh I'm going to choose to prioritize this one over that one then you make the other team feel like their work doesn't matter. It's just really hard. One of the challenges of building AI in 2026 is that you have to figure out how to get the people to work together even though you're only building one thing at the end of the day. And so this particular technology has been really instrumental in helping more people work together to make Neotron stronger.

Interviewer: Fascinating. So it's just as much a technology question as a human organization question.

Nvidia Representative: Exactly.

Interviewer: Okay. Fantastic. Let's put a pin in this and get back to this in a second because it's a fascinating topic in terms of the post training that you just alluded to. One of the exciting things that you all did in the context of Neotron is also to publish the data, the training data. Does that include per industry data for specific reinforcement learning tasks?

Nvidia Representative: Yes,

Interviewer: That's the beauty of a conversation like this today where you guys can actually talk about those things. So where does one get the data from for post-training reinforcement learning focused efforts? Right obviously one of the key questions in the world today is that LLMs or AI systems have become great at coding and great at math. Like the next big question is can they become great at law and consulting and then all sort of different domains. And part of the black box of closed models is like how people go about doing all of this? Where did they get the data from? To the extent that you can talk about all of this, I'd be very curious about how you guys have gone about it.

Nvidia Representative: It's not an easy question to answer because it is quite complex. But I would say we rely on a number of things. One is that we do purchase data from companies that are building data sets that you can purchase. And to the extent that we have the rights to redistribute or to open up that data we do as part of our Neotron data effort. With Neotron, we are trying to be maximally open with the data that we release because our goal is to support the ecosystem, right? Our goal is not to be the only model out there. And we love it when we hear of other models around the industry that are using our data sets to make their AI stronger because that means we're succeeding at our job to keep the ecosystem thriving and growing. Now we also are big believers in synthetic data generation. We use an enormous amount of compute running language models on our own systems to create synthetic data that then helps our models be better at solving problems in specific domains and we release a lot of that data as well. Now it's of course not very straightforward to do this. AI is always garbage in garbage out. So you have to work really hard to make sure that any synthetic data that you create is actually adding value, that's actually helping the model generalize and solve problems more intelligently. But those are the primary ways that we go about building our data sets.

Interviewer: Since we're talking about post training and RL in different domains, just curious to get your thoughts on where we go from here in terms of generalization. So just to build on what I was saying a second ago, like the industry seems to be marching from coding and math which are domains with verifiable rewards to different industries. Do you think that this is where things are going and that the industry as a whole is going to be able to cover those next few domains as efficiently as coding or math?

Nvidia Representative: Coding is really special because it's a very intellectual exercise that created a lot of economic value which then meant that we had an enormous amount of tokens that we could learn from as well as tooling that allows us to verify whether our models are actually solving problems. So coding is always going to have a special place in our heart and something that I think AI is going to continue to get much better at because we have this special relationship with it. With regards to other domains I think what I'm excited about has to do with significantly more diverse environments for AI to learn in during reinforcement learning. I believe that reinforcement learning is such a general form of teaching an AI how to solve problems. We're just getting started at figuring out how to apply that. And I think as our environments get more sophisticated, the AI then learns more understanding of the problems that it's trying to solve as well as the implications of the actions that it can take. Then it becomes much better at at

Speaker A: actually um solving those problems. When I look at the the uh environments that we're using today, they're still uh fairly simple um all things considered. And I think um uh that's going to become significantly more complex and diverse over the next few years.

Speaker B: All right. So you mentioned uh you know making 500 people work together and I I said we would get back to it because it's so interesting. So just uh taking a step back like tell us about the research organization at uh Nvidia like how is it structured? How how does it all work?

Speaker A: Well uh Nvidia is not structured according to an org chart. um we have one but it's not actually the best way of understanding how we work. Um uh my team for example is not part of the official NVIDIA research team. My team is actually part of the organization that builds the GPU and my team is not the only team building Neotron. There's probably 10 teams around the company that have significant involvement in building emotron um in different parts of the company in in enterprise software um in uh uh our AI software um uh division the part of uh Nvidia that actually designs the GPU also significantly is involved in in building Neotron. Um so there's there's so many different teams that that have to work together. Um we um always like to say that the mission is the boss um rather than uh the organization. But um what that implies is uh that people have to figure out how to work together um which is challenging um in the sense that humans are naturally tribal creatures and it's uh not natural for us to um be friendly with people we don't know very well or trust uh co-workers that we don't have you know success working with in the past. And um you know uh actually the name Neimotron uh reflects that. We had um the Nemo team which was building software for AI and the Megatron team which was building uh primarily focused on systems research for um uh for building large language models and um you know we decided to work together and then start calling our projects Neimotron reflecting you know sort of the um uh the collaboration between these teams. Since then, Neimocron has has dramatically expanded. There's so many more teams that are part of the effort.

Um, and it's really important um that we have structured it uh in this open way inside of Nvidia. Um, you know, we are inviting uh volunteers from around the company to come help build NVIDIA's AI. Uh, we think it's very important to the future of the company and, you know, as that vision continues to develop, more and more people want to join. That's fantastic. We're really excited about that and it means that we then have to figure out uh how to organize the work so that everybody has a chance to contribute and feel heard and feel like their ideas um are are uh you know fairly evaluated um on the on the path towards impact. Um uh we we have a a formal process for doing that. We have an internal website where people um share ideas and then those ideas um are assigned to um one of 25 different um leads that are uh you know over various parts of building Neotron. Um they interact with those ideas. Some of those ideas get further developed. Um some of those ideas get deferred until you know the next uh the next time we we go around building a new model. Um but we're trying to build Neotron in an open and inclusive way um uh so that u you know we can really come together as a company to build it. I think um you know organizations that figure out how to collaborate to build AI succeed. Organizations that struggle with control over who owns the AI tend to uh waste a lot of effort. Um and so Nvidia's success and Nematron success I think is directly proportional to our ability to collaborate something that I care deeply about.

Speaker B: Fantastic. But you you mentioned uh earlier that um despite the fact that you work at the you know number one undisputed leader in in GPUs you all as a research organization don't have all the GPUs that that you would want in the world. So like how does the allocation of GPUs and compute uh happen? And is that based on how promising an idea is or early success? Do you give GPUs, withdraw GPUs based on success?

Speaker A: It's a really complicated question and it's it's obviously a a difficult problem for everyone in the industry to figure out how to allocate their their compute. Um uh inside Neotron uh you know so we we have a budget uh for Neotron. Um and inside Neotron we allocate compute based on what we think the needs of the project are. We have a um a hierarchy. So we had a we have a set of programs and and in inside of each program we had have a set of projects and each of them put forward their requests. Um and then you know we have a two-week cycle where we review requests and we review the budget and then we make decisions um in kind of a hierarchical way and then um you know uh compute gets decided that way.

Um now having said that um this is something that I think we can still do better at. Um uh it's hard when we're making decisions about compute allocation because um every researcher is convinced that their idea uh could change the world if it just got a thousand times more GPUs attached to it, right? And they they might be right. It might actually be true. And yet we're running at the limit. We don't have a thousandx more GPUs for every idea that that we have. We we have to operate within uh the limits that that we have. And so it is um a challenging process. We try to incorporate as many people's um perspectives into that as possible so that it's as much as possible a shared sense of uh understanding maybe not agreement. So there may be times when one project feels like it really deserved more GPUs because the impact of that would have been so high but it didn't get it. We hope in that circumstance that they have an understanding of why some other project did get more GPUs and why that was considered more of a priority during this particular allocation round um for the company. Um so that people can at least uh understand you know that there's a reason uh for the allocations that we have. Um uh having said that um you know this process is always improving. There's always more work to be done to make this more transparent and more fair. Um and and then of course u my my number one is just to get more GPUs so that um you know we can also fund more things because I would like to do that too.

Speaker B: How do you balance useful research with great exploratory research?

Speaker A: My belief is that research needs to be bootstrapped. Um it research is a chicken and egg problem. So um it is always the case that every researcher believes if I just had a lot more resources my idea would change the world. Actually, it's important that researchers feel that way because if you didn't feel that way, you wouldn't have the conviction that's required to go do something crazy and new, right? So, you have to believe um and and so of course um uh you start with that belief. Uh but then how do you translate that belief into something that other people can understand, right? That other people are willing to invest in. Um this is what I call the the chicken and egg problem, right? Because like once your research idea is obviously good and impactful, it's easy to get resources. But how do you get it to be obviously good and impactful without those resources, right?

So so the way you solve chicken and egg problems is by bootstrapping. This is an iterative uh problem solving approach where you do something small. you get some sort of signal about this is a good idea and you tell people about that and then you ask for just a little bit more and um if people saw like oh yeah that you know that experiment turned out pretty well that's pretty intriguing we should probably do a little bit more there um then you're on track right and uh that that um over time you know iterate a lot iterate quickly iterate many times uh you can bootstrap to you know finding significant resources for your idea and also usually attracting more people um to come along with it uh on the way because they have a chance to see that this idea is going to change the world and then they want to be part of it.

Speaker B: Is that how the moonshots at

Speaker A: Nvidia got started as well over the years whether that's in AI or otherwise? So it was bottoms up somebody coming up with a good idea versus Jensen saying this is what we need to do.

Speaker B: Well, you know, Jensen has lots of good ideas, too. And so, the company is very responsive to his ideas, and that's important as well. But Jensen very explicitly says all the time, this is a company of volunteers. You know, each of us is here because we choose to, we could be doing something else with our lives, but we choose to be here. And so, you know, we tend to make decisions, especially for early stage research, it tends to be very bottoms up, because, you know, it's sort of an invitation like bring your best ideas. Let's figure out, you know, what are all of our best ideas and then we'll take a step from there.

Now do we sometimes have top-down ideas that are important for the company strategy? Of course, you know, of course. NVFP4 pre-training is one of those, you know. So, we decided as leadership of the company, we're going to really invest in FP4 hardware. Now it's time to go invent some optimization algorithms that succeed in using it. And, so, we told the team, we didn't say to the team, you have to work on NVFP4 pre-training. What we said is, there's an opportunity, we're making a big investment, and if we can figure this out, it will be significant for our company. And then we let the people who are interested in that work on that and as a result we succeeded you know.

So it is a balance of like bottoms up and tops down, but it always has this bootstrapping feeling even with something like NVFP4 where there's a significant like strategic top down component, the actual technical solution which is very intricate and complex and has a lot of moving parts that came from the researchers themselves and you know that's my belief is that research always comes from the researchers themselves. You can't tell research exactly how to go solve a problem because then it wouldn't be research, it would be engineering. But in a world of AI where the most important problems we have to solve all have this research component, there needs to be freedom for researchers to innovate if we're going to make progress.

Speaker A: Listening to everything you're saying, I'm struck by how entrepreneurial the culture at Nvidia still seems to be. So like it's a very large company so I'm sure there's also politics and you mentioned the tribal instincts like I'm sure all of this is happening but especially given you know how long the company's been around a phenomenal success the fact that people have been making a lot of money internally it still seems to be very entrepreneurial bottoms up driven maybe meritocratic is that the right takeaway?

Speaker B: Yeah I mean one thing that's very unusual about Nvidia is the tenure of its leadership. Jensen Huang has been running the company for 33 years, but he's not alone. There are a lot of other very senior leaders in the company who have been there for three decades or longer. Including my boss and these people remember what it feels like to work at a very small NVIDIA and they know what it feels like to work at a very large NVIDIA. They have a shared sense of ownership for the company.

You know Nvidia is a place we often say no one fails alone and the point of that, that's just a statement of fact right. You work at a company it's a one company you all succeed together you all fail together. You work in accelerated computing, accelerated computing is the composition of thousands of technologies. If any of them fail to deliver acceleration the value is destroyed. It doesn't matter whether the chip is great if the compiler sucks. At the end of the day the thing that you're selling is time and capability to researchers that are trying to build the future of AI. And if they don't get that, it doesn't matter whether it was the transistor or the math unit or the compiler or the library or the networking or anything else along the way that failed to live up to its expectations. The whole thing in composition fails, the whole value is destroyed. And so we have a deep understanding of that culturally at NVIDIA and it is something that motivates the way that we work together.

Speaker A: Maybe to close the conversation, I'd love to zoom out, get your take from, you know, the perspective of somebody who's like as deep into all of this as it gets about where things may be going. So like who knows in a few years, but I don't know in the next year or two maybe there's some visibility. I read somewhere that you're not necessarily a big singularity kind of a person. Is that fair?

Speaker B: True.

Speaker A: And what why is that?

Speaker B: Well, I think that intelligence is just so incredibly multifaceted. You know, I always think about this question like if a company were to be looking for its next CEO, would it find the next CEO by looking for somebody who won the International Math Olympiad? Probably not, right? Even though like it's incredible for people like I could never even compete in any way at the International Math Olympiad and those people are amazing, right? They have just incredible brilliance. That's not the right kind of brilliance to run a company.

If we look for example at other aspects of our culture that are really important. For example, musicians. What kind of intelligence does it take to become a hit musician? Don't assume that it's all luck. It's not. These people are working hard and they're very smart in ways that I might not understand with my PhD, right? I might not have that kind of intelligence.

And so when I think about intelligence, I think it's just so multifaceted and so contextual. You know, it really depends on the situation. It's not just about raw intelligence. Raw intelligence is kind of like the horsepower of an engine, but an engine running without wheels doesn't go anywhere, right? So the impact of intelligence has a lot to do with the context that the intelligence is put in the harness, the platform.

And so when I think about that, I think you know the singularity is although it's an attractive idea, I think that it's really a wrongheaded idea because it doesn't really take into account these other factors. So I believe that artificial intelligence is going to continue to develop at a rapid pace. It's going to unlock significant capabilities for people in every aspect of our world economy, people doing every kind of work. I'm very excited about the opportunities that it's going to bring.

I am also a little bit concerned with how we're going to manage the transition. So I do think that transitions are hard for humans in general. Like we're conservative generally. And you know there is going to be a lot of change. This is a profound change in the way that we think and the way that we work, the way that we learn.

Ultimately I have faith in our ability as humans to figure it out. You know we've done it in the past. This is who we are. We build tools. We build external organs that help us solve problems. You know we have an external stomach. We call it kitchen. It creates enormous value for us. We can eat things that we couldn't eat without a kitchen. Right now we're creating an external brain. You know the implications of the external stomach were pretty profound for us as a species. They led to agriculture which led to organized societies the way our cities are built. So we think about what is the implications of an external brain pretty profound. Nobody actually really knows. But what I do believe in is the power of humanity to solve problems and to learn and to incorporate new technologies in ways that benefit us.

I also believe that the problems we face as a planet all require more intelligence. Every single one of them. Whether that's inequality or climate change or you...

Brian: know any of the other structural problems that I think are very worrisome that we face. The solutions to those are going to require invention and intelligence. And what that means for me is that the only kinds of tools that we can really create moving forward are going to be AI. Because the problems that we face are all about intelligence. And regardless of the technological approach to solving those problems, the solutions will always be called AI. And so that makes me hopeful for the future but also you know somewhat respectful of the challenge that it is going to bring to us as we try to figure out how to live in a new way with this new external brain. But I believe in our ability to learn and to change. And I think ultimately this is going to make our lives better.

Matt Turk: Do you guys feel the AI backlash that seems to be forming internally? Is that something that you all perceive think about? And if so, do you think it's a communication problem that our industry may have? You know, in particular, given what you just said about all the obvious potential of AI,

Brian: you know, I'm always worried about the way that the public thinks about technology and interacts with it. It matters a lot. And it is definitely the case that societies that want technological advancement have more technological advancement than societies that don't want change. So I think it is actually important to think about it. One thing that's interesting about AI is that I believe it tends to be much more accepted when it is part of everyday life and then at that point people stop thinking about it as AI. It's just, oh, this is the tool that I use. Like, do you care whether it's AI that's helping you route your car when you ask the map application to help you drive somewhere? Like, I mean, there is actually sophisticated AI that's going into that. But you're not really thinking about that, right? You're just using a tool. And so I feel like people's acceptance of AI comes with experience, right? The more experience we have working with it, the more we learn how to work with it productively, I think the more comfortable we become with it.

Matt Turk: Great, Brian. So, it's been a fascinating conversation. Maybe as a very last question to make sure we cover it, I want to make sure that we talk about safety. What is the state of safety currently? And where does open source and closed source sort of fit in the safety conversation today?

Brian: Safety is on everybody's minds right now. You know watching the Fable release and the way that the government interacted with that I think is a consequence of concerns about safety about these models. You know, they get stronger and stronger and then they could be misused. And you know, there's different approaches to thinking about safety and trying to define safety. I have maybe a slightly unorthodox opinion about this, which is that I think open technologies are generally safer because there's more sunlight. You know, when more people are thinking about the safety of a technology and evaluating it and then contributing to making it safer, I think that's inherently safer than having a small group of people being in charge of safety for everyone else.

I also think with artificial intelligence because it is really about ideas. It's really about exploring ideas in different ways. That diversity is more safe than monoculture. And what that means is that there's going to be different beliefs. Like diversity isn't just about the easy stuff. Diversity is about the hard stuff. Like when people have deeply felt disagreements, they really really totally disagree with each other. Making it possible for people to explore their ideas in a diverse way I think is more safe than trying to create a walled garden where you know certain ideas are considered safe and certain ideas are considered unsafe.

And you know this is controversial in today's AI environment which I think is interesting because we've had hundreds of years of tradition that speak directly to this. You know, in the United States, for example, we have laws about freedom of conscience and freedom of speech. And you know, it's not because we didn't consider for thousands of years would it have been safer if we didn't have those, right? We tried that. We actually tried having a monoculture about like these ideas are safe to talk about, these ideas are safe to believe. And we found that to be much less safe than a pluralism where we officially don't take a position about what ideas are safe. We actually found that is much safer as a society to support diversity than it is to try to keep everybody safe top down. And so I believe that open technologies for AI are inherently the safest way of building AI.

Matt Turk: All right. Love it. Controversial take to close the conversation. Brian, it's been fabulous. Thank you so much. Really appreciate your spending time with us today.

Brian: Thanks for inviting me.

Matt Turk: Hi, it's Matt Turk again. Thanks for listening to this episode of the Mad Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you on the next episode.