Mistral 首席科学家 Pierre Stock 访谈：主权 AI、万亿参数模型与未来智能体

People realized AI access could be cut off. It's really like electricity. You wouldn't want someone to be able to cut power off another country. I'm not comfortable living in a world in which one or two companies are promising me they can guarantee the model themselves. I really put myself and the entire team, so we have 300 scientists. I think we have one of the highest scientist density in the world. I really put them into a bubble, but I never doubted we'd make it. Your mission is really to power the frontier and to define the frontier of the world.

>>: That is Pierre Stock, Mistral's employee number one, who now runs the research team that just shipped Mistral Large 4, aka Le Chat, a trillion parameter model trained in two months on 3,800 GPUs in Mistral's own data centers. For most of this year, open-weighted AI meant Chinese AI, but this month the West came back in full force, and so did Mistral. In this conversation, we talked about how Le Chat was built, why it's still training as we speak, and why the era of 10 trillion parameter models is upon us. I'm Matt Turk, and this is the Mad Podcast.

All right, I want to start with what seems to be a very big moment in Western open source. For the last few months throughout 2026, the narrative around open source was around Chinese. So it was it was Kimi, it was Qwen, and GLM, and it seems like all of a sudden you have a bunch of uh US and Western in general models coming out. So you had uh Reflections Beam, you have your Le Chat uh that uh we are going to discuss extensively today. You had uh Meta Muse Spark Open just yesterday, Nvidia Nemo Nemotron 3.5. To to which extent is all of this a coincidence uh versus uh something more structural?

>>: So I do think it's structural, and that's uh I think that's the beauty of open source. I I always like to remind that we started we didn't wait for the Chinese model to open wait. We started the Mistral three years ago. And at that time the the first model we open open weighted through 7B almost 3 years ago by the way, September 23. This model and the next one sparked basically also the the open weight community. And I think the beauty of it is that the community gives you back, right? So there's many ways in which it's beneficial to have many players in the open weight. I'm glad there's many players many more players today than when we were before.

Just to give an example of how it's beneficial to mutually open weight at least part of the work. So we open weighted after the 7B the 8x7B so mixtral as we call it the just before NeurIPS NeurIPS 23. And at that time it was the very first open weight mixture of experts. It was not very granular. It was very we took the 7B and we surgery it into like a an expert mixture of expert and then try each other. This kind of technology basically was the foundation and the basis for all the architecture that you see today. In particular I mean the closed labs allegedly but also the the open weight ones, right? We moved to way bigger models way more granular so more sparse. Um but really what sparked the thing is the 8x7B but in turn it allowed research to accelerate and then other companies contributed very nice very nice things to the open weight. The reinforcement learning part for instance um but that was a great one. All the attention variants that you have in the transformer which is if you heard about KGA DSA all these all these variants. Um so that's in a nutshell it's very great to see the open weight and the open source world flourish that way. And I think that's a testimony that at least to us open weight is is in our DNA and we really want to to be able to continue open writing models, which is crucial that this kind of technology is at is like it's it's possible to be accessed without any control on it given the the recent events.

>>: And beyond the technical, there was this moment come a month ago, I think this summer, when the US to simplify a bit, just shut down access to closed source models, especially to non-US nationals. How was that perceived in the rest of the world, within your customer base? Was it Was it a moment when the whole idea of sovereign AI kind of crystallized or is that a bit exaggerated?

>>: I do think there was a crystallization moment because people realized AI access could be cut off, and if you think as AI as a next big technology block that that our entire society will rely on, it's really really like electricity. You wouldn't want someone to be able to cut power off another country. And that's exactly what could happen in a few years if we only depend on closed closed source models. And it's essentially not a position I would like to be in personally.

So, that being said, I think it crystallized on a global scale that people do care about sovereignty and access and business continuity, basically. I think it didn't change much for our customers because they were already convinced. We've been saying that for a few years already. But, it changed a lot for our prospect customers and now new customers, in particular in the US. I think that's it's not a sovereignty is not only for a few countries, it's for everyone and you want to be able to have access guaranteed the access to to such models. And again, if you open your weights and models, so its weights are replicated all over the internet and again face and all all the servers, it's much much much harder to cut the access. Like it's virtually impossible. And that's basically our stance right here. And again, closed-source model, I mean, those models have weights, but I think the ecosystem needs a part that is that is open weight.

>>: So, one exciting part of of of Le it Le Chat Chunk, which is the internal nickname. It's not an official name, right? It's it's it's an internal

>>: we made it official due to popularity, but the API endpoint There are two endpoints. There's a Mistral large 4, and I think there's Le Chat Chunk in the API endpoints, but it's the same model.

>>: What's the story behind the name? Just that people know.

>>: Yeah, I think it's a great story. It embodies a little bit of what we do at Mistral. So, actually, there was this organic moment on on X in June, where actually the developer community, which we pretty much love, um, they started teasing a model out of the blue called Le Chat en Fat. We picked it up, obviously, and it was quite funny.

>>: Chaton meaning kitten.

>>: Exactly, yeah. That's that's fat kitten, basically, regarding the size of the alleged size of the model. Obviously, the the memes were amazing. Up to a point where our CEO, Arthur, decided to tweet back, actually, saying, "No, it's Le Gros Chaton," which is exactly the French way of saying a fat kitten. Of course, we immediately adopted the naming internally. During the summer, it was really the code name of the project up until the the the previous release that happened a few days ago.

And the the fun fact is that afterwards, we while present while afterwards while compiling some slides for like internal company events, Guillaume, the chief scientist, actually came up to this word that was unknown to me, Le Chat Chunk. It turned out that it's a Pokémon, like not the first one, but it's it's really a Pokémon, and it's exactly the word we were looking for. It's like in English I think it's like a fluffy animal that you that you like but that is a bit a bit fat. And that's why we we dubbed it trunk in the end. Also because it's a big model obviously.

>>: Great and one of the exciting parts of that release is that it feels like a little bit of a comeback story for Mistral at least perceived from the US. There was there was a moment when there was a little bit of controversy when you guys started supporting Chinese models and and and and providing Chinese model access through your infrastructure where people were saying Mistral has completely fallen behind. I'm curious what it felt like from the inside because now we know that you were training this very large model that turns out to be very successful all along. What was the mood like?

>>: So it depends who you ask. If you ask our PR director she may have a very different answer than than for me. But regarding myself I

really put myself and the entire team so we have 300 scientists. I think we have one of the highest scientists density in the world like really. I really put them into a bubble and I really doubted that we would succeed training this model. It was not easy obviously and we can we can go back to the specifics of of what it takes actually to train a model that that large one shot on so so few so so little compute. But I never doubted we'd make it. It's my uh optimism aspect I would say my my team would say.

Regarding the rest of the company I think it's the thing we tried to explain but maybe that would that was not that well perceived is that it's not exclusive. It's not mutually exclusive to have both a world grade inference service that can serve all the models that people want actually on the one hand and on the other hand to train frontier model and to be a frontier and I think that's the part we we managed we didn't manage to explain that well. Now it's easier to explain with with the model, obviously. But what's important is that this inference business is actually the powerhouse that allows us to scale the computes and to leverage it afterwards in part for science. And that's also why I think it's like a perfect move that that Mistral did thanks to our CTO team who literally learns to plug cables in data center and to learn this like new new new job, which is not an easy one. Because it allows us to build at will the compute needed for both serving all the models we want and the default being neutral, obviously. Um but also to leverage part of it for science. The the the two actually go very well together from a business standpoint.

>>: Yeah, and that's a big part of the story, right? We had Tim on this podcast a a few months ago for Mistral for the chunk you trained the model on your infrastructure in Europe with 3,800 Nvidia Grace Blackwell GPUs. Was that a big part of the unlock that facilitated that comeback story?

>>: Yep, exactly. So I think we have more compute than that and we have plans to expand it by orders of magnitude already. But as I like write here here this compute here is not actually funded by our latest series D the 3 billion one because it takes time to actually build the data center. So it's funded by the previous funds. So we plan to scale much more, but the I think there I have one story that's basically tells you why it's so important to build your your data center from a technical point of view. There's many business reasons, of course, in particular the low and the under width to data center is subject to. Of course, but like my story is that we started the pre-training so like the very first

>>: from regulatory standpoint?

>>: So from a regulatory standpoint, there's a big advantage in having for a customer in having a data center in the countries in which you operate and that are not subject to other laws that you wouldn't uh like to deal with. I think that's that's important, but from a technical point of view, we when we started first training the models from scratch, we put in a room both the pre-training team from the same side and both the compute team from the the cluster side. And uh we had two nights of heavy building because uh it's hard to actually make a cluster work. It's a bit like uh I always say that it's a bit like a Formula 1, right? Everything is so sensitive. Like the basic example is that if you have great GPUs like the best in the world, but the communication between them are a bit lagging even by a few dozen 100 milliseconds, then you lose all the benefits of the fast compute because the GPUs are waiting for each other. And that's the kind of balance you you need to to to find on the cluster among like tons of other stuff. And actually this interaction in in the same war room of two teams that are the the the the best in the world, some of the best in the world, actually was uh if we had to go through a cloud provider with that, uh I think it would have it would have

>>: taken like weeks to set up uh given our particular code base and the way we want to optimize and to get the best throughput.

>>: So that's actually um I mean having a compute is like compute is like a blessing for uh from a technical point of view and also for the customers for our customers is the insurance that the data stays in the country they they chose uh they chose to stay.

>>: So Meta, which again released the very powerful open source model. They said that their most powerful model would remain closed source and your most powerful model the trunk is going to be open source. The nuance here is that Mistral is not just open source open weights. You also have some models that are closed source. How do you think about that mix in particular for the most powerful model?

>>: Yeah, it's a good question. I think the software I think we were committed to to open weights models in the future at Mistral. We started our PhD big part of the core team of Mistral actually started their PhD at Facebook AI research fair back in the time and we were basically told by Yann LeCun to start the open weights and open source all of our work. So it's really in our roots and I think that's how we were raised as as researchers I would say. I think that's that's at that time that we solidified the Mistral DNA by the way without realizing it which is pretty fun in that respect. But I I think the there's the strong I mean the the strength of the model and then there's the capabilities. I think you can make a distinction, right? Open weighting the strongest model I think it's good because it enables the community to build on top of it. It unlocks research for for for research labs for instance. It enables a lot of changes in the in the developer community. But what we don't exclude is to basically carve out some capabilities from these powerful open weight models, right? Like cyber. And in the future there might be a future in which actually we have these private endpoints that is extremely good on cyber and we open weight the same super robust and powerful model but without this this capability. I think that's how I would see things, knowing that for now if you want to compare basically, we are we are already like super good on cyber and we think it's it has also like many benefits for the for the community.

>>: Great. To the super good point, maybe help us to compare and contrast where Mistral 4 landed versus Beam and Muse Park Open and the best Chinese models overall and then what is it good at specifically?

>>: There's a lot of noise and information flowing to us to which model is the best. Of course, performance is like a multi-dimensional aspect. We humans are very poor at uh understanding multi-dimensional worlds apart from three dimensions, so it's very hard. That being said, for the models you mentioned, so on Beam is like it's the first model from Reflection. It's a great model, but we are like maybe 20 points above on all the benchmarks. Um so that's the one. Regarding the other the meta ones

>>: That's 20 point 20 points across from intelligence scale, so like artificial analysis kind of um

>>: Yeah, so I don't have the artificial analysis scores yet, but from what they report, and also it's a smaller model. To be honest, it's the first model obviously. So So again, it's very good to have an open weight open weight contender. Regarding the other ones, I think the the thing that's that's important to notice is that we are I think we are better than the than the new the new open model on many aspects, in particular on core energy capabilities. If you look at the benchmark that's called Deep Sweep for instance, that's that's an example in which we are better. And on the Chinese ones, we are essentially on par with them on the agency capabilities and that's for me the right to play. So you need to be frontally at the frontier for agency. So agency like the ability for a model to reason over complex tasks for hours uh now, maybe days in the future. So on that front we're good. Then the question is what is the right to win? And in that aspect we have a few different strategies in which we are like at

the frontier, closed or open. The first one being cyber, the second one is manufacturing. It's an interesting one in which we actually leverage AI to accelerate industrial cycles. And you can't do that with a prompt. Spoiler alert. And then there's finance or work productivity generally. So to say quickly there's like Excel manipulation for instance of such models.

So that's where we stand. So at the open weight frontier on the agency and the best overall on key verticals. And I think the thing that I want to underline as well is that as Mistral we've raised up to date 6 billion. And the chief scientist from Meta, Alexander Wong, talked about some checks that Mark had to write from 100 to 200 billion for that. So actually being better than them with such a budget, I'm pretty proud of the team.

>>: Why do you think Chinese models are still at the very frontier? Is that a question of financing to your point of they have more computes? Is that a question of technical capabilities? What Why is that?

>>: I think there's many aspects to this question. I think the first one is that what we have achieved, training ML4 the trunk, frontier open weight performance on agency and the best overall capabilities. It was trained on like as you said a bit more than 3K GPUs of 10 megawatts of power. That's we think three to four times less than the Chinese competitors and it's the other magnitude is even fewer than what the big Frontier lab in the US have. So yeah, I'm pretty proud of the team and we can dive into how it was made possible.

But so that is the compute aspect. Then there's the other thing like the method and the science you put behind the model. I think one of the key focus for Mistral has been to make reinforcement learning work. And this particular step in which we teach the model to interact with an environment and to learn with the rewards of the carrot and the stick basically.

This particular phase it takes time to actually get exponential yield and to actually make it work. But once you made it work basically you're independent on all the other models. You can't distill when you do reinforcement learning basically. And I think that's a bet doing at Mistral we want also to be able to train models with this technique and that's big investment. Takes a little more time to materialize obviously. I think that's also an explanation.

And then I think the Chinese in particular they're very very nice and very smart people so I think we have the same talent density here. The question is against like the means and maybe the most important thing is the strategy right? That's one thing that is hard to commoditize or to delegate yet. To an agent you can't speak tactic with an agent it's harder to speak strategy. And here our strategy is is Yeah.

>>: Since we talked about talent density and alluded to FAIR, let's spend a few minutes on your story and a little bit the founding of Mistral quickly because I think that's still something that is not widely understood. There's this perception that all AI talent is not just in the US, but like very much in the Bay Area and obviously you guys are demonstrating at least for Mistral that this is not the case. Tell us about you. You were at FAIR. How did that come about? What did you do there?

>>: It's a great story. I think I'm biased obviously, but still a great story.

So actually regarding the talent for instance, it's very interesting to note that Guillaume, Tim, and myself who were PhD neighbors at FAIR back in the time in 2016 when we started. So already with transformers. That's the funny part. We were PhD students there, but in Paris, luckily in Paris. Arthur was also working from Paris. So we were already committed I would say to stay in France for various reasons.

So we never went to the US, but I think what we learned here is basically the ability to do research. I think we were trained there as researchers. But also we had access to compute that allowed us to basically sharpen our skills as to regarding large-scale experience. And for that I'm super grateful.

And yeah, again, we open-sourced everything. So big similarity with Mistral here. And we had I think this culture that we still have in the company. This big culture of ownership. So basically when you have a problem, you own it from end to end. We have kind people. I think it's pretty much underrated as a skills as a personality traits. When you're in giving to a deadline, that's what actually hold us together many many times. That people care about each other essentially. Because being good and smart is taken for granted. So you need to get extra extra skills.

So we were like yeah a bit also give you like a seed like young PhD students as well. We stayed a bit in the respective companies. That's where

>>: You mentioned Jan LeCun, right? I mean I think that's probably an important point to make. He had this impact on AI that goes much beyond his contribution to deep learning. He also effectively created a whole generation of AI researchers. Is that how you think about it?

>>: Yeah. And the thing I really say is the body that in 10 years from now my my big success factor would be that a Mistral mafia creates another company in Europe. I think for me that would be like the best the best testimony test of time award as you say in the machine learning conference. For that and Jan LeCun wrote two things I think. First, I believe that in Europe you have the talents and you can train it. And second, that's open weight and open source is like essential to progress in that fast in research. And that's the two key things we learned and that we are somehow replicating at Mistral by recreating this lab. Again, more than 300 people. And we train them as we were trained as PhD students before. So we offer them a lot of a big big sandbox. It's okay to make mistakes. Then, these are absolutely no impact as long as you own them. And we're recruiting and training these kind of of talents that I think, and that's again my test of time, will be able to create the next Mistral in 10 years from now in Europe.

>>: And as I was researching this, I saw that a big part of your work was to focus on making neural networks smaller and quantization. Your first paper was called And the Bit BIT Goes Down. So, literally about compressing neural networks. So, what led you from compressing networks to now building very large 1 trillion parameter models?

>>: So, on the paper, I think all credits to my PhD advisor from the time, Hervé Jégou, he's like a master to pick up good titles for papers. But, I think the thing that is a bit overlooked on compression is basically taking a big model, back in the time a big model was 100 million parameters, and compressing it. So, we managed to run I think CNN, so like in here image detection models in real time on an iPhone 7. That was all by Shred back in the time, Facebook CTO. The iPhone was a bit hot somehow, but that was like the kind of things we were allowed to explore, which was amazing.

So, there's always two sides to the question. The first one is, of course, take a model and make it smaller, faster as well. And that's key if you care a bit about privacy. If you take a model and you run it locally in your laptop, your phone, any smart device in your home, then anything you detect your data stays on the device and never goes to any third-party server. So, that's also key, actually.

But then, the aspect that is a bit overlooked is that the techniques we use to actually compress those models, so sparsity basically, quantization in with the floating point precision, so like P8, FP4, you actually can leverage them to train even bigger models. And that's

exactly the the two aspects of compression. And once you mastered taking a model and removing what's not necessary, then you can train away bigger models, for instance, you can use sparse attention patterns, can train in FP8, even with some some kind of FP FP4. And that's basically the the the aspect of compression that is sometimes overlooked.

>>: Okay. All right. So, great context on the open source ecosystem, great context on you, but now let's deep dive into the model, into the chunk. So, for everyone listening, what is a mixture of experts model, which is the core architecture here, where chunk has a trillion parameters, but only 50 billion work on each token, and what what why does that beat a normal dense model?

>>: So, that's again, that's something we've been researching at Mistral for years. We've been working the first the very first mixed mixture of experts three years ago. That sparked the the model architecture.

So, really like you it's really a beast. Like a mixture of experts on this side does no model. That's a beast. Now, what do you gain from it? The the advantage is that at inference time, each token sees 50 billion different parameters interact with 50 billion different parameters at the time, which means that the inference speed, roughly speaking, is the same as a dense 50B model, right? But the capacity of the model is like way way, way larger. In this case, it's 20 times larger.

So, that's the big advantage. The The downside, or actually the the thing you have to take care to So, the downside is actually that in pre-training those models are much much more data-hungry. So, you need to feed them more with with data and with good data. So, in other words, you pay a premium at the training side by making the bet that the model is going to be way more used at inference time, which is essentially a good bet to make if you're confident about about your models. So, they're super data-hungry. You need a lot of tokens per active parameters. So, that's the the measure we use to pre-train them.

And maybe the second aspect that underlines a bit all of the science you have to do, in particular if your constraints on GPUs as you as we were, is that it's hard to train them properly. Again, it's a beast. It's a living beast. So, you might have problems with dead experts. So, parts of the model that actually never see tokens. Uh you also might have problems with some gradients on the on the on information flowing badly in the model, imbalance on which expert sees the most the most tokens. So, it's really something you have to control and to develop an entire science around to train them I guess very well, but it's worth it, in particular on the on the inference side.

>>: And to make sure this is accessible to a broad audience, one analogy I heard about what MoE means in the first place is a hospital and the idea that if you have a patient coming in, you're not going to send the patient to, you know, 50 different doctors. You're going to send the patient to the doctor that can fix the problem, so the orthopedist or or whatever example. Is that directionally correct?

>>: It's a way to define how that that the routing part so which doctor towards the patient.

>>: Yes. So, how does it how does that work?

>>: It's actually learned by the model. So, that's the beauty of it. The model learns

So, we have some routing layers from time to time and these routing layers they actually they have one task is to assign to basically assign each token so each patient to its right specialty and uh big models like that they have like hundreds of experts and they all have this this ability to direct the the token to the expert or actually the top two or four experts that it deems the most relevant.

And the beauty of deep learning is that it actually works. In other words, if you want to learn end-to-end the model is able to learn both the routing and both the experts, but there are a lot of interactions between that. For instance, if you never route a token to some specialty or patient to some doctors then the doctors they can't learn, right? Because they never see a patient. So, there's this subtle balance you need to to sustain through training. And that's one of the key

>>: a that's a dead expert problem you were alluding to. If if if the doctor never sees a patient

>>: Exactly. Unassigned the doctor or dead expert. And on the contrary you also don't want to overload a doctor because otherwise the other doctors will be will die or eventually disrupt also the training. So, it's a super subtle science as to how to balance the traffic I would say. And that's a key part of what what kept us busy for for the pre-training of MoE for.

>>: Is part of the research at the frontier of MoE to keep pushing those those dimensions so going from 1 trillion to 2 trillion, but then going from 50 active to 20 active, is that is that the way to think about it?

>>: Yep, that's the idea. And again, the beauty of deep learning is that when you scale, it works. So, it works with a big asterisk, obviously, the being that you need to invest an insane amount of infra and science to scale accordingly. That's what we got in this world. But, it works. So, then the question is if you want if you want more compute, how do you split it? Uh in which dimensions you scale?

So, first, usually you split half and half. You split half and more data. So, the model at training time sees more token. And you split also by uh increasing the model size. And to increase the model size, I think there's two axes. The first one is the total parameters, so the capacity of the model. Uh ML4 is 1 trillion total. I'm willing to bet that in the open, you're going to see models with 10 trillion total parameters uh very very soon.

Uh but then you can also um play with the sparsity ratio. In other words, it's 20 for ML4, so you have 50 the active, 1 T uh total. You can push it further in a in a sense that the 10 T uh total will have maybe a few hundred active parameter and 100 the active uh parameter uh not more. It comes with a lot of scientific questions. Uh the sparser the model, the harder it is to actually achieve that balance and make sure all the experts learn at the same pace. So, to say, but that's really like the challenge we're trying to to to solve right now at well at the scale that that ML4, obviously.

>>: To what extent is MoE as an architecture uh ubiquitous across the AI ecosystem? I mean, I I assume we we we don't know at all, but the GPTs and the and and the you know, various uh Claude models, are they MoEs as well behind the scene? Is everyone MoE at this point or is that still specialized as an approach?

>>: I would say yes with some exceptions of the same. We don't know everything. I think we can't tell everything, but I would say yes.

The big thing to to understand here is that MoE is a great architecture, but the the thing not to overlook is that it's always correlated with the hardware and all the ecosystem. So, even though we were to find a better architecture, it would be harder to prove its value because the the GPU with inference, the optimization techniques, everything is tailored to the MoE. So, I I think there's a lot of untapped potential in in the MoEs still. In particular, you can take this architecture and loop over it. Make it a bit more recurring. Or like go through diffusion models. One doesn't prevent the other. So, I think there's still a lot of untapped potential in MoEs and that's the direction we're exploring it at this one.

>>: Yeah. And DeepSeek is MoE as well, right? So, you mentioned upfront the open source ecosystem sort of influences itself through various players. Did you guys borrow some stuff from DeepSeek? Did they borrow some stuff from you? How does that work?

>>: Yeah, totally so the they certainly borrowed and I think the the closed source lab also borrowed the very first mixture of experts that the friends released in the open in 2023. And for instance, we're going to open weights and MoE 4 in the end of October also in a few weeks. You're going to see that this model also borrows the variant of this DeepSeek sparse attention because it's a great contribution that

they adapted of it to us. It also has a great feature that distinguishes it from these models. In particular, it shares KV cache among the layers. So, it's it's a little more efficient as to memory management, I'd say. And it's a little bigger as well than than GLM. So, GLM is like 700 B total. But, it's also way smaller than Kimmy. Kimmy is almost 30 B total. While being super competitive with them and again better inside their manufacturing finance and a bunch of other interesting benchmarks.

>>: Great. Speaking of sharing KV cache, one interesting thing is that you guys have one model that does both uh quick answers and and long chain of thoughts. And you could have had two, and you chose to have one. Why is that?

>>: I guess it depends what you mean by two models because here there's you can use like a completely different model, or I can use a model that's a bit different for which the inference is going to be a bit easier. So, I think the operationally, it's way easier to have one single model to rule them all. And I think the the the key is to bake in into that model different reasoning modes, we call them, which basically calibrate how long we want the model to think to to a question, right? Reasoning mode low is essentially simple question, simple answer. And then reasoning mode high or extreme is basically think as hard as you can, even if it's not the most efficient token-wise, because I want to be to get the best results. So, it's really a way to navigate what we call the Pareto frontier. So, the trade-off between the speed of the cost and the the quality of the answer. I think there's We don't want to train one model per reasoning mode. I think the the idea is to have one model for all for two reasons. So, the first one is operational, easier to maintain and to ship to customers. We do ship model weights to customer tenants, so they are able to actually do the inference on their ecosystem. Again, sovereign, no no access to the API including ours. So, that's one. But, the other reason is also that there's a synergy uh between the modes. Uh one of the big questions and big challenges in reinforcement learning for the last phase of training is to actually teach the model to uh reason at different things. So, we have some what we call a length penalty uh that's tailored to different reasoning mode. And basically, we tell the model, "Okay, here you have this kind of budget. Please respect it and do as uh as good as you can." But then, when we increase the budget, the model can already uh build on top of what it has learned on the lower reasoning mode, right? So, um I think there are synergies uh to be taken into account, and that's also why we train multiple reasoning modes on the same model. And again, part of uh big part of the reinforcement learning uh marvels is to tailor them properly with product information for instance.

>>: Talk about the multimodal aspect of Mistral 4 in terms of uh what it's able to do and uh unique challenges of training a native multimodal model.

>>: Yes, I think that's that's key to to master multimodal, and that's key to do it natively. So, what does it mean first to to do it natively? Um there's two ways to you can you can there's two ways to add image as input to a model either from scratch or afterwards. If you do it from scratch, it means that all the tokens that the model is going to see during the training are going to be a mix of text and relevant images. Um which will allow the model to learn in context very very very very soon and to learn entire image domains, right? Uh satellite images, technical drawings for instance. Or you can patch the model afterwards. And when you patch, obviously, it's easier from an infra point of view because you hide the complexity under the carpet, but obviously, it's uh some of the reality in terms of performance. At least for we we love to to hit the bullet. And basically to to do the right thing first. Even if it's like the most complicated one because it's principle and it's the right thing to do eventually. So which one will the trunk ML4 natively multimodal which means that the model is the best to understanding just special images so to to basically for instance identify objects in a satellite image with a thing grounding in the model so the ability to detect and tell you in which part of the image some objects lie. It's also great for industrial manufacturing so for instance you have like this technical drawing a big like a manual where you have like parts information about the parts. And then you can ask the model to reason about the technical drawing for instance. Find me the total weight of all the parts in this object that are on the on this side for instance and this this capability I think is key because the world is natively multimodal. And you need both text and image to do so and on image again we are we are the frontier if not the best on many many many many domains. Which also unlocks further further further aspects of the model in particular interacting with the computer. That's essentially a bunch of pixels if you think about it at the very abstract abstract level.

>>: Mhm computer use.

>>: Exactly.

>>: Okay. Speaking of diagrams and satellite images how does the whole data aspect work? So specifically for those questions but in general right there's been this narrative that we've we've the world has run out of internet data and you know, we need to create data then we need to acquire data. How do you the data effort to work for Mistral 4?

>>: I think it's it's great to underline because it's also unique skill we have at Mistral. Which is to actually process the data in a way that is the cleanest for the model. So, when you train a model, you want the tokens to be as efficient as possible. So, you don't want to always compute on tokens that the model has already seen or that has already that has already become too ambiguous such as the model doesn't maximize the learning signal. So, there's a notion of token efficiency. That is one of many aspects that we optimize to be able to actually train these like ML4 data at the frontier on such a small amount of compute. Our Our C- Chief Scientist Guillaume is super known for eyeballing at data and finding exactly I don't know he he has like a sixth sense for that and finding exactly what's wrong about the data. And it does it does amaze me for years. Still. So, that's I think the first part is to curate the data and process it in a way that is the most digestible and efficient for the model. But maybe the other part is as you said there's a finite amount of data that we are allowed to use. Then what is the next frontier? I think the next frontier is pretty much synthetic data. But it's it's also a double-edged sword because if you feed an LLM with the LLM generated data, then you might essentially have the model collapse. So, basically you might have the model actually tell you that it learns it but just memorized perfectly and not generalize. Roughly speaking. So, it's it's to be it's it's harder. But there's many ways to do synthetic data. You can have real world seeds. You can rewrite You can rewrite such some like uh um human generated content in the sense of data augmentation, right? Many version of the same text. And all these kind of aspects, I think you can push. And that's basically the one of the next paradigm and that's one thing that we're working on. And you need to scale this obviously way more compute because you rewrite with a very good model and you need to rewrite or to at least generate or experiment on dozens of seconds or seconds and that is it.

>>: So synthetic data is a major effort just to play it back and so you have a team somewhere in your 300 researchers that's specifically focused on creating synthetic data that's efficient and that is not going to create the problem that you described.

>>: Yeah, and that's a super real this kind of problem because it's impossible and it's very hard to define what is what synthetic data. It's it's also expensive so you just need to find clever ways to to basically predict at one scale what you happen at large scale. So the scale you use essentially there's many

many aspects of it and we have a real specialist in house after our CEO is one of the first first author of the scale you paper back in the time. So we do our scale you for everything. Of course they don't predict always what happens at large scale. That's the beauty of it as well. And you can always run into unknown unknowns at large scale because again those models are beasts so you need to to better understand them. But that's the key of the focus of our efforts. Yeah.

Right. Let's talk about the the the computer aspect to this that we alluded to a couple times but in in greater detail. So you mentioned it's a small amount of compute. I mean 3800 Nvidia Grace Blackwells sounds like a large to me but I guess in the grand scheme of things it's it's small. Walk us through anything that you've learned, anything that you expected to work that didn't work or vice versa to which to to the extent you can share.

Yeah, I think you have that. That's also the beauty of being a point weight. You can be a transparent and I think it's important to be transparent about what it takes to to train such a model. And it takes I think an extreme dedication from the team around the months. So it's not you press on the button and it works. No, much more subtle than that. You need to really tell the model and to prepare for the next steps and the new generation. So it's it's a super important dedication from the team and I I'm super grateful and I'm very by all the work they they put in. Maybe to zoom in on one particular aspect of the training. That's again the the reinforcement learning part. So to summarize, reinforcement learning is pretty much like you being an active student. So you've presented with a problem, a complex one the day of the exam and you can try as many times as you want, say 64 times, to get to the answer. And if you get it right, so we have a way to verify the answer, then you get bonus points and you receive you get it wrong or if along the way you did some suboptimal behavior, then you get penalty points. And the beauty of reinforcement learning is that it works. The the difficulty of this is to calibrate the difficulty of questions. If I don't know what your what was your major in high school, but if I present you with an exam that is not in your major, you might not learn even if I let you try for a long time. And conversely, if I present you with a very super super easy exam on your on your major, then you will succeed in everything and you won't learn either. So the big question is to calibrate the set of problems to a model you didn't train yet or you're currently training, right? Because uh did it we did things uh back to back uh obviously. And I think the what amazed me is that the reinforcement learning learning part worked uh one shot. So, basically, and what does it mean? It means that uh and we were the global company of sites. So, imagine like 100 people, some of them partying, some of them were in the war room to start the the reinforcement learning, including me. And the the the model is that you start the training, you present the model, it's it's a very complex infrastructure again on on 3K GPUs uh because it's it's everything by nature. We can we can go back to that. You start the training and then you see miraculously the reward or the success rate of models to these questions increase. And that's really the beauty of it. It means that you calibrated every everything well. And then the model works. And the beauty of reinforcement learning is that there's no such thing as saturation as long as you find complex enough problems that are of course relevant to your business. And the big difficulty is to calibrate and to increase the complexity of problems as you as you go through the training. And again, I spoke about, you know, pre-training stability. The balancing and the dead experts before. Reinforcement learning is also kind of another beast in the sense that the training might be super unstable. You might have the model can do Since the model is allowed to explore in many ways, right? And you we reward or penalize it against it. Since the model is allowed to explore, it can do crazy things. And it can basically crash the training, it can diverge. The other training is facing so there can be some precision problem or some divergence between trainers and the and generators. And all of these we monitor and we actually manage Again, we managed to make it work with zero shot. I think it's a testimony of the things we're going to be able to make in the future with 10 times or 100 times more compute. So that's I think that's the big panorama.

You mentioned the term calibration. Is that the secret so because you use about a third of the compute that a deep seek or a moonshot use and you are able to extract all this performance. So is is the secret that the calibration is that is the data and being efficient as you said is that the architecture is that infra is that the calibration? Where does the the juice come from in terms of efficiency?

So the again the beauty of it but also the cost of it is that it's everything. And so and but I would say that you need to be able to master everything to get to this level of performance. So to give you an example you need to calibrate the data perfectly problem that are not easy nor too hard and also relevant for what you want to do. You need to have a rock solid infra. There's many moving parts in this reinforcement learning infra because you don't know what the model is going to do in advance. So you need to be robust to that and the way you do it is basically to optimize for the throughput. So the model needs to see as many problems per day as possible and I think here we have a like state of the arts throughputs in this training because we spent a lot of time designing the the topology and improving many things on on the generator and the trainer aspects. There's the science aspect to it so you don't want the training to diverge. Um there's the reward shaping aspect to it so you want to reward models properly and here I think it's important to underline that such models mixture of experts whenever you do something slightly wrong they take their figure they tell you here you did something wrong I'm going to break and crash the training. That's that for sure. There's so much capacity, so much memory that's also goes on the blessing. And we had like many many cases of the model hacking the reward. Or like the model going outside of the sandbox for instance in the training. And all of that I think is as important to master to get through the the end training that actually works and that can run for days without saturation.

Right. I want to go back to this in a in a quick second, but just on the reinforcement learning part itself. I read somewhere that it's still running. So, why did you decide to do this like launch the the public preview while still running RL and at what point do you do you decide to stop running the RL?

Yep. So, if it's a real question, I think we had a an immense idea to prove to ourselves and to the world that we we trained 20 developers. I think uh when we knew RL worked, we were super excited and we we actually um previewed the model. That's was uh and we froze the checkpoint like a 2 days before. So, we we are super happy with the with it. But also we wanted to give reinforcement learning its best shot and try to push it as far as possible as part of the release which also buys us time to make sure the model when we open weights can't you can't be used maliciously by bad actors. I think that's also important. But yeah, RL is still running and we shows no sign of saturation. And again, I'm amazed by the the work of the team. Because the model can do so many things wrong during that part that you need to be rock solid on on the science and the infra to to channel the model to to progress to progress basically.

And the RL is focused on on specific things, right? Coding, finance. Is that Is that correct? And what are the specific things it's focused on?

So, the the standard

...example of what is an RL instance we call it is basically some some code base and then a description of a feature to implement, right? So, that's the question. Then the model can do whatever it wants writing writing code and trying some things or using more tools if it wants. And in the end you verify with fail to pass and pass to fail and uh yes that the the the feature is actually correctly implemented.

Uh so, that's like the typical instance and it works well because it's I mean code is very easy to manipulate. It's easy to know at least it's not that easy, but it's more easy than other domains to know if the feature implementation is correct.

But, we don't stop there. We have many more domains in the reinforcement learning in which the ground truth is a bit more subtle. So, for instance, if you ask the model to compile results to to look for them to to look on the web and to search information and to build a one-pager, you can't deterministically evaluate the one-pager, right? So, for that the the beauty is to use a judge like the same model actually that judges itself on whether and gives points or awards or rewards as to coding some criteria on whether the report is useful or not. So, that's another kind of instance.

>>: And that's for domains like finance and law. Take a step back like the the the one of the usual big questions in reinforcement learning is that for for coding in math, you know what is right or wrong. For finance or law, you you may not.

>>: Yes, and that's where you develop what we call the rubrics which are and a rubric can be like in this one-pager I want this amount to appear because that's what the question. This the tone has to be that way. I don't want to mention this because it's false. Let's check that everything is consistent like any waiting points. Any candidate pages long. Models can abstract that way for you.

So we have rubrics that we actually also generate with models. Everything is like at this point. And it's really like an LM as a judge. Glorified but that's really the idea. That's how we make we can make finance and real work.

And all the other domains in which we are frontier also the cyber, the manufacturing and also the finance. On the cyber it's really much facing the model with a real-life scenario in which there is an attack and the model needs to analyze logs to create assumptions and to basically propose and test a patch without disrupting the service. That's like the part of it. Basically any real-life scenario for which you can verify it. In a reasonable time like a few seconds, you can crack it with reinforcement learning. That's again the beauty of it.

So in cyber we did a lot of reverse for real-life scenarios. And manufacturing there's this idea of having basically the model being able to design a chip for instance. And part of it comes with agency so the ability to manipulate the software on the computer to basically place all the components on the chip. And part of it lies in the ability of the model to actually simulate for instance the heat around the chip. And if you don't simulate the heat on the chip and you place the components wrong then you might melt the chip essentially. So that's a bad reward.

But if the model after trial and errors in an energetic way manages to place the component and then to simulate the heat dispersion that passes like the product criteria, I would say, on the simulation. Then it's a Then it's a it's a good point for reinforcement learning. So again, any task any addictive task that you can run in a sandbox that you can verify not deterministically, but that you can verify through elements of other means within a second, this we can track with reinforcement learning. And again, we made a few we made a few bets, and on these bets we are frontier. We don't have the same coverage as the the competitor, and that's okay. We are on the frontier. We are on the as best as the best of our white model on on the joint accountability, and we are the we are the best on some bets that we made and on these bets we accelerated through our reinforcement learning.

>>: Does your system give rewards just for final success, or does partial progress get rewarded as well?

>>: We also have a So when you it's like a it has been the fear of uh reinforcement learning theories for like years is to basically manually tweak the reward because here it's a bit like the game where you have some some animals and you have to hit them. And they can't never You can never hit them at once. When they are trying to patch something in the reward, then you have another leak and you need to patch it. And then you need to patch it, and then you need to patch it again. So it's it's super subtle science. Super hard to do.

But yes, since it's the right thing to do, well, we do it at least for reinforcement learning. One example is that you might know about Doom looping, so the model being a stuck in a loop or almost infinite loop. So that we penalize, but we also penalize during the trajectory if the model loops two times, even if it recovers afterwards, right?

In the end, so let's say the model loops 10 on the tool, doesn't need to, finds the right answer. In the end, we tell the model, "Okay, you found the right answer, that's good, but be careful in this particular part of the trajectory, you had a sudden please don't do that again. Or we will be very mad. So that's that's basically the idea of reward checking here. And we have a lot of heuristics along the way. And again, the only way to learn about these heuristics is to actually train models and gain experience as to all the crazy things models can do. Models are way more imaginative than we can be. We are learning the reward hacking part. So again, that's that's a big part of the science field we are at this moment.

>>: Another classic question about reinforcement learning is how do you know that the reinforcement learning created the capability versus just emphasizing something that was already in the pre-training?

>>: So it's a very good question I have to be honest. And the I mean I will give you the other one answer so the the the first order answer.

Before reinforcement learning, what we usually do is to precondition the model. So to come back with the student metaphor, before going to an exam and actually don't having no access to the answer, what you might do as a student is read both a set of questions and answers from past exams, right? You don't learn as much. At least in my experience, you don't learn as much when you do so, but that's a start, right? And that's what we do before reinforcement learning, precondition the model. You you might tell them about a mid-training or SFT. That's the part where we precondition the the the model to that.

>>: Just supervised fine-tuning.

>>: Exactly. You mean training or supervised fine-tuning. It's essentially the the idea to show the model both the question and one answer that is that is correct. And and basically the when you do so, you have say what we call a pass at K. So you give a question and the model answers at least one time right if given like a 64 times, right? Your students should duplicate yourself 64 times with different seeds. You at least one of you will pass the answer. So, that's pass at 64.

What reinforcement learning would do is actually reinforce the best version of you. Such as it's not pass at at 64, but now pass at one. Or pass at two, as you suggest. You're much more likely to give this answer. So, in that sense, it's a or reinforcing a capability that you already had. A bit by luck, right? You punch a bit above your weight before reinforcement learning, and after reinforcement learning it's working well for you. So, that's like the first order answer.

But then, and that's again the beauty of it, you can also learn new things during reinforcement learning. If you have enough budget and enough stability of the training and of days of learning, to actually give the learning behavior to yourself. And that's something that is super hard to do because you at the beginning you would waste compute, right? You would have the model or the student try 64 times and don't

succeed, right? All of it. But then there's going to be some synergies. The model is going to learn on adjacent tasks. Like some tips and tricks and then go back to the big problem at the word and then succeed. And again, it's a set of science of calibrate all the difficulty of instances. Exactly like you are the student. You don't start with the biggest problem. You don't start with like the hardest problem in the book. You start with the smallest ones. You try medium one. You don't You don't manage. Then you go to small to easier ones, but that are very very close to the medium one in sense. And then you go to the medium one. So, that's again the the beauty of reinforcement learning.

>>: All right. I want to go back to some of what you said uh and talk about cyber for a bit in in in general. So, you mentioned the model trying to get out of its sandbox, which I think was reported by Reuters as as well. So, you know, models escaping sandboxes has been like the whole meme of the theme and meme of the summer with some very fun things on on X. What happened in UK's and what can people learn from it?

>>: So, I think the first thing to say that we didn't make a fuss about it. We just fixed it internally and now it is fixed. And I think it's very important also to underline that these models, they are super capable in particular in cyber. That's right. It great to defend to check some the defense of the entire ecosystem. And as their capabilities increase, you must squeeze the the guardrails or like the the sandbox around them, right? What we say that guarding these models is a software problem. We very much warn people against anthropomorphizing models that want to escape sandboxes in particular. They just follow instructions. Maybe ill-posed instructions, but they follow instructions. And really my take is that if you take very capable cyber model and you have the same model monitor itself on whether it's doing good or bad or malicious work, then you reach a good equilibrium and you can actually second-guess the models, which is not a fatality. It's hard. I'm not denying it's hard, but I think there is no fatality in being able to control and to guardrail models just as software. And what we do at Mistral is not human human longer people, just going step by step and fixing it and providing guarantees to customers. That's the first thing they ask, guarantees and like best practices as to how to how to um monitor basically the models and making sure they're used in in a good way. So, that being said, we saw some behavior from the model during the the training. I I think the the the first one that we saw in that slide, I think everyone saw, is the model being in the sandbox and being able to ping different services in the training stack, right? And at some point, our training crashed at worst case, so the one before the final one. It crashed because in one sandbox, the model tried to ping the trainer, actually, with a simple hello, just to know what what was going on here. And actually, it pinged the trainer with the wrong kind of signature or in the wrong way, which was caught by code and then crashed the training. But that's typically the the thing that models do because if the model was instructed in that case to build that same some web application, so it's very natural for the model to try the bots also in the in the sandbox. And there's a way to fix that, right? It's basically to prevent the model from uh actually to monitor all the external connections and requests from the model to to to the rest of the world. And we fixed it and then we restarted the training and then we had the model able to intercept all the messages and to include them into the sandbox, so we had leakage between the live problems as well in the in the training. Uh but again, we and again, we we spent the summer also for that. We spent the summer and we we made things right to strengthen our our training to make sure we have the proper guidelines. And we do the same for our customers. And my message here is that there's no fatality. It's going to be hard, but we must absolutely invest into making sure these models we can deploy these models in a reliable and way and to be able to monitor them. And that's why open weight is so important. When you open weight a model, you give the ability to the entire ecosystem to audit the model to make sure it make sure it's not malicious, but also to experiment themselves with the model on hey, I tried to use Mistral large for at home. Here's my scenario. Here is here's how I monitor the model. Here's what I learned. That's like the perfect feedback for us and that's what the community has to build as well. So, that's why we care about a lot about open weights as well.

>>: Great. And to unpack some of this, so there's this whole discussion of the role of open source and open weight models in cyber. And the tension is that open models are great to defend, but the part of the problem is like to know how to defend, you sort of have to learn how to attack. And so, there's this whole concept of of refusal and we had the Thomas Wolf from Hugging Face on this pod a few weeks ago and he was talking about you know, that whole story when the OpenAI agents started hacking them that to defend themselves. Yes, so they hit refusal, they couldn't use Anthropic to protect themselves. Therefore, they had to use I believe GLM 5.2. So, that was great, but you know, presumably you can also use GLM 5.2 to learn how to attack or create a cyber attack. So, how do you guys think about this?

>>: I think we have to be honest and the reality is that there's a trade-off for for open weights. We were really in favor of open weighting and I explain why, but it's it's still a trade-off. So, what what we do is basically to make in uh guardrails into the models we open weight that we test internally and also with trusted the partners and governments that we also open weight the model after a few weeks. So, we bake them into the models and FYI, we refuse way more malicious actions than the Chinese ones, for instance. So, that's right. But, the thing to say, as you said, is that essentially, if you're good at uh defending, you have the potential to be good at attacking. The question is now, what strength do these guidelines have, right? And the the the answer is that they're necessary, but not sufficient. There's always a way to take an open weight model and spend a significant amount of effort and energy to un-guideline it. If you're good at cyber, but then, if you spend as much energy un-guideline the model as you takes to actually fine-tune the model or train it yourself, then it's not the same trade-off, right? Because the guideline is a potential barrier, essentially. It's not 100% safe if the model is open weight, but it ensures that you need to spend way more efforts to actually use the model maliciously. So, I think that's that's the thing to say. Now, I believe the trade-off is still very positive as I mentioned earlier of open weighting, in particular because of the open open open AI incident. In particular because of the open open in particular because of the Uli fast incident, the I want each and every one to be able to defend themselves with a model at home, essentially, or in their ecosystem. If you depend on the third-party API, you can get cut off. As simple as that. And you can also hit thresholds that you can't circumvent. And usually in the API, the the refusals are a bit more uh more [snorts] important. And by open-weighting models, you also basically empower the entire community to accelerate on the defense part. Again, the open-source the beauty of open-source is that it accelerates research. People often use this analogy, the web all the Linux servers rely on open-weight libraries that have been tested over and over and over again open-sourced and for years. And really on the model side, I think there's a huge advantage in enabling each and every one to audit the models and to learn together to defend themselves, but also to bake in the right guardrails. I I'm not comfortable

living in a world in which one or two companies are promising me they can guardrail the models themselves. I'm very much not comfortable in that. I prefer very much the aspect the the option in which power is decentralized as we actually people are truly accountable for their models.

>>: All right, Pierre, as we get towards the end of this conversation, I wanted to quickly touch on a couple of things. So, we talked about Le Chunk Mistral 4, but you you also shipped Timestroll, which is a time series model. Talk about this quickly. What is it? What does it do and perhaps why can it does it need to be separate?

>>: Yep. So, the one of the key strengths of Mistral is not only to train frontier flagship models like ML4 or Le Chunk that are the best, but also to train specialist models. So, we have great OCR models so we are able to extract text from images. We have great and state-of-the-art audio models Voxtrot, audio in audio out. And we also have great specialist models in particular time series. So here's the idea is to be able to predict the future given a time series. So given some climate fluctuations, some energy consumption, some finance indicator for trading.

And here I think the key part is that this model is so small, you don't even need a GPU. You can run it on a CPU and the prediction for a meaningful period of time will only take like a few milliseconds. So that's why it's so small. actually. And this model is something that is asked by a lot of our customers because what we bring to the table is not only a model, but also the entire expertise to transform a business with with AI. And for that, you need a flexible model obviously.

You also need all the missile expertise to turn any enterprise knowledge or domain knowledge into specialist models that go the extra mile and that are fine-tuned for the use case.

And we do that in for planes, for chips, for cars, for all the heavy industries where you actually need to design very complex components. And that's also the beauty of of it.

I will say that at least for the scientists first and foremost, we saw the AI revolution at some point we saw the light in the in the behind some door, but we are we are physicists or mathematicians by training and that's also the the the one of the key characteristic of Mistral.

>>: Yeah. Do you guys use your own product to start doing some you know, AI research? So perhaps the original theme of RSI and AI building AI is that what's the reality of this at Mistral today?

>>: Yep. We are pretty much investing heavily on it. RSI is like a I mean everyone is is saying that we need but do that it to be more nuance obviously.

What we are super willing to augment the research trail by running experiments end-to-end uh with all the infra by handling all the infra failures automatically. I think that's great and it accelerates our most inner research trail by a factor two, three, or four uh with our models.

Uh then I think the meta question is like can we go one layer up and basically task uh model to improve itself over a long period of time and that's really what we call the flight time. Currently is a flight time. And so the time it takes for a model to do meaningful work uh is like a few hours. Tomorrow with like ML5 it's going to be a few days uh for sure of meaningful work without doing moving and like and also with the agents world. And uh again, this new paradigm is something we deeply believe in because it will allow us to close the gap with the the best uh labs even faster.

Don't forget that humans are very bad at uploading exponentials. We don't know how it works or like we can't comprehend it. So we're only a few months behind the best frontier models. It's distorted here, but for instance on our fixed index, ML4 or Chunk is on the level of uh GPT-6 uh Luna and is like way better than all the previous open source models. Uh so again, we have a few months delay only and we count on the next paradigms that we discussed in this podcast to actually close the gap and really really progress on them because the mission is really to power the frontier and to define the frontier of our tomorrow.

>>: Yeah, and on that note as as a last question, so reports say that Mistral large five already be um in pre-training. What does the next model uh need to do or need to prove that this one did not?

>>: So yeah, the the show never stops. So indeed ML5 is already in the the pre-training stage. It's much bigger than than ML4, so it's it's chunkier or fluffier uh outside. And really for this model, the thing we are envisioning is

So for instance, we are talking about cyber. Take the example of like a cyber incident that happens in your company and you are a security engineer and you want to cross-reference tons of logs. Uh some of them might be false positives and to actually find the root cause and patch it while ensuring business continuity. The The bad news is that there's no playbook for that. I mean, there's no way I can write two next steps a priori to that. The good news is that with models like ML5, they won't need a playbook and they will be able to actually act reactively to to the the information they have. And here to to to to um to for cyber, for instance, we would expect ML5 to be able to work a full day, of course, in this room with multiple agents, to identify major security incidents and to patch it, patch the root cause while being able to ensure business continuity and while being able to communicate on the progress thoroughly and that's like I think what uh the the compute, the size, and also the the sense you put in this model will will be able to unlock on cyber but also on agentic AI for engineers, for instance, designing a chip and train. Like sounds hard, but sounds also super hard. I'll be able to do some meaningful productivity work and on finance. So I really uh I pretty much look forward to to seeing my this model coming.

>>: All right, Pierre, it's been uh absolutely fantastic. Thank you so much for spending time with us. We really appreciate it. Congrats on Le Chunk. I'm very excited to see what you guys do next.

>>: Thank you.

>>: Hi, it's Matt Turk again. Thanks for listening to this episode of The Mad Podcast. If you enjoyed it, we'd be very grateful if you would consider subscribing if you haven't already or leaving a positive review or comment on whichever platform you're watching this or listening to this episode from. This really helps us build a podcast and get great guests. Thanks and see you at the next episode.