Host: We got so much Gemma 4, Gemma 3, Gemma scope, Med Gemma. Give us the TLDR.

Omar Sanseviero: Yeah, so yeah, Gemma 4 is just out. This is the most capable open model we've released so far. We really tried to compact as much intelligence per parameter as we could. Bring all of these multimodal capabilities. So yeah, that's Gemma 4.

Host: So one interesting thing, you have this thing with effective parameters, not active parameters. Can you explain what it is?

Omar Sanseviero: Yeah, so pretty much in the traditional transformer architecture you have like this big embedding layer, right? And this new architecture is is more of a small change in the transformer architecture, in the transformer block. Pretty much we add a per layer embedding. So at every layer we add an embedding table. What is exciting is that you don't need to do like the full matrix multiplication. This is pretty much a lookup table. So the Gemma 4 model is a E2B. That means that it effectively has 2 billion parameters loaded into the GPU. It actually has almost 5 billion parameters, but those 3 billion parameters can be in the CPU, they can be in the disk, which means that you can do inference extremely quickly. This is just a lookup table.

Host: And what's the con? Why don't we Why don't we always do this? Can it scale? Is it open research? Like you know, it seems very Okay, if I can just offload half the parameters to CPUs.

Omar Sanseviero: Yeah, so pretty much here we did lots of quality experimentation and this is really optimized and designed for like on device. And when I say on device I mean like running in a phone, Android, Raspberry Pi, and so on, right? When you go larger you usually want to compact more. You want to have more like dense architectures or MOEs. So this this research This research decisions were very helpful for these small small use cases.

Host: I met Cormac and he was telling me that I apparently in China the super apps are shipping models in the app bundle. For inference and just like use among all their super app. Assistants. Yeah. And I don't know is is is that like a target use case for you guys?

Omar Sanseviero: Yeah, so actually if you install like if you buy a pixel phone or a high end Samsung, they come from with a Gemini Nano and Gemini Nano is baked into the operating system and Gemini Nano is really built on top of Gemma. So last year we released Gemma 3N which was this architecture really designed for phone use cases and they use a Gemma 3N with some additional training, some additional adaptations to make the model good for like traditional on device use cases, right? So pretty much when you buy like these high end phones, you can already use a Gemini out of the box.

Host: What goes into shipping a mean line model like this? Like Yeah. What what's the behind the scenes?

Omar Sanseviero: It's complex. The Gemma team is actually relatively small. We have like two or three PMs, we have one marketing person and then there is our like engineers and researchers working on shipping this. Of course there's like the full training part, we how do we do the post training, distillation, post training techniques and so on. What is quite exciting is that once we have the model, then we collaborate with a bunch of open source partners, right? So for example, we work with a Lama CPP, Olama, MLX, Hugging Face, vLLM, Nvidia, AMD. So we have almost 50 external partners for every well for the Gemma for lunch, which has been the most complex launch. And also internally, we collaborate with a bunch of different teams. So, think of Google Cloud, Vertex, Vertex models models as a service, ADK, uh and then Android as well, right?

Host: What's the difference? When would someone want to do that versus just using Gemini? Outside of course Outside of the obvious, you're offline or you want the privacy.

Omar Sanseviero: Yeah, yeah, it's mostly offline use cases. Right or if you Yeah. Offline or privacy, like if you want to have all of your development set up locally and you don't want to send any code to to any API, you would use that.

Host: Can we talk about the multi-modality side? Any advances there that you want to highlight or you've been getting good feedback on?

Omar Sanseviero: Yeah, so Gemma 4 was built on the same research as Gemini 3, which pretty much means that we benefited from all the improvements that happened with Gemini 3. Multimodal wise, the smaller models can understand audio, images, and short videos. So, 30- to 60-second videos and and audios. For us actually quite long. And even the on-device, like the 2B, 4B, 2B can do very good multimodal. Yeah, for audio we have a speech recognition, we have a speech-to-translated text, and then a bit of a speech understanding. So, you can do like a ask questions about an audio file and so on. So, use cases that are very optimized for like on-device phone use cases. And then on the vision side, we also improved things quite a bit. So, we have object detection, pointing, captioning. We do not have image segmentation, which we know is like one thing that many people have been asking us. But otherwise, like for many things, we do support that. The other thing we do not support yet is video with audio. So, we can understand like video input or audio input separately, but if you want to pass like in the same prompt both a visual part and the audio part, we still need to do some improvements around that.

Host: Gemma is quite important for the multilingual aspect as well. So Gemma supports these 140 languages. Uh You did a lot of work on the multilingual order the tokenizer.

Omar Sanseviero: So the tokenizer has been pretty much based on the Gemini tokenizer. It's extremely good. So independently of the Gemma capabilities, if you just pick base Gemma model and you fine-tune for an additional language, it actually works extremely well.

Host: I do think that research and engineering are closer than people think.

Omar Sanseviero: Yep. Uh there's I mean there's research engineers. Yep. And Mechatronics is probably the easiest single easiest way that engineers can get into research if they want to. Yeah, I think I mean in in big part like so many researchers are doing ablations, right? Like they are just moving the pieces around and seeing what works and what doesn't work. Of course there's like a branch within research that is more much more like architecture design and like a much deeper. But there's lots of very like empirical experimentation and seeing what works, what doesn't work, moving things around, which for me is much more engineering rather than for like research unless we are like writing new activation functions maybe, but Yeah, and something that is cool that is happening is also how researchers begin to adopt some of the cool agentic tools now. So for example, within the team we are building a skills to do experiments and ablations and evaluations and how the research team can use all of these agentic tools as part of their research process is also quite interesting.

Host: Do you have a take on auto research?

Omar Sanseviero: Every AI wave has an auto ML wave and this is the auto ML wave of this wave. I don't know like with Karpathy experiments it's been quite interesting to see like how things are evolving. I will I don't know what's your take on this. Things are just cooler when he does it. I do think some some part of this is you're just speed running experiments agentically right? The agent The coding agent is more autonomous you can actually go to sleep and it will do the things that you would have done anyway. So you're just kind of automating things that you would have done. I see it differently. I think okay it will be a very exciting time if we have a move 37 from an auto research. If you make an impactful discovery that someone wouldn't have thought of right? So there's a side of okay I have these ideas go run them in the background that's fine. But the the The side is actually when you're shooting off not just paths that you wouldn't have thought about, but you know, trajectories that people wouldn't think about and they work and you make new discoveries. That's the very exciting thing. I think when you have more approach to just token spend and send off, you know, hopefully that becomes possible. Yeah, I do think the next generation of fine-tuners will not be like I mean will be people that are not coding at all, right? Like 1 year ago we had to write like our own code that with transformers or on slots or or whichever library of your choice. I do think as we like keep evolving like most people will be fine-tuning with a couple skills, right? Like Hugging Face has a skills, like all of these libraries have skills. They will just uh prompt the agent to kick off like some experiments and see what works, what doesn't work. And honestly, it's a it's a good middle ground right now. Like all the tools you've mentioned, they let you fine-tune in minutes. You don't need to know what's happening in the order. Yeah, so I think that's where like the direction will be like people that just want to do fine-tunes to improve their capabilities for certain domain or like add some like new behavior. They will not be coding the fine-tuning code, but of course if you want to do like deeper research in the architecture, my hunch is that most likely uh this will not be like automatable at least in the next 1 or 2 years.