So, Gradium is a is a new company we created back in September and basically it's about powering uh real-time uh voice applications. What we mean by that is we focus on the technological layer for voice interaction. We don't make voice product for specific verticals. We don't make voice agents. We make models and infrastructure to create transcription, synthesis, translation, operate them at scale and help people create products around them.

A bit of backstory about this company. It's a commercial spin-off from QI lab. So, it's a nonprofit lab Lauren, myself and a few others created two years ago. So, in particular, we introduced the first speech-to-pech models. Uh the first speech translation systems, uh realtime TTS, realtime speech to text. Our models uh have been used as the backbone for uh quantities uh sizes and pretty much all the current audio generating models out there. So we decided to create uh a company to you know make actual production ready models ourselves. Our team is mostly uh focused on engineering and uh science. So as Matt said most of the people are from deep mind meta and quantitative finance. So it's a small team really focused on the infrastructure part.

So basically the current setting around voice AI is the following. So until recently voice AI was mostly about offline content. So that would be just reading an article on a news website or generating audio book. The most exciting part is much bigger than that and much more about personalized and interactive content. In one big example is gaming. So more and more we see that uh video game studios want to integrate interactive NPCs and a lot of experiences around voice. Live stream is also a case where uh there is the technology is really lacking at the moment. So there are some uh solutions for dubbing for example but they mostly run offline. If you tried or were a victim of the uh dubbing uh on YouTube that sometime turns on automatically you can hear you know how uh it's not really faithful to the original voice. So there are a lot of opportunities around uh being able to integrate voice into live content. uh robotics is also an obvious one where uh an interaction will be by nature individual and needs to happen in real time and need to be scalable. So basically right now the landscape has changed because when you generate an audio book you can have a solution that is slow and expensive because you amortize it because you generate an audio one that is consumed by the lot of people. Now when the audio is just a part of a personalized experience or an interaction it's generated for a single person and just and then thrown away. So it must be very fast and it must be very affordable as well.

One of the things that we have been working on is personalized content in particular being able to customize voices with exceptional quality. So here for example is a a benchmark we run against 11 labs and uh basically we have better voice cloning and it's not only means that uh if you take uh like someone speaking in a natural formal tone we will replicate their voice better but we can also capture kind of very specific tones and styles. One example for that I think that is a good one is a gaming uh project we are working on with a big mobile game uh developer. The main idea is to be able to get a a personalized esport commentary in your game. Uh so you know it will be like when you play FIFA or these kind of games uh you know it's everything is scripted. You have a set of MPC files in the in the directory of the game that are played one after the other. Here the idea is that every gamer will have a personal experience based on the context. You have an LLM that analyzes the state of the game in real time and creates the scripts for two fake commentators. Just with 10 seconds of voice from uh two actual commentators, we are able to create uh something that really sounds like an actual commentary.

Hello and welcome Brawl Stars fans. We're following Panda YT who's currently sitting in a strong second place in the Angels versus Demons event and looking to climb higher. He's queuing into Trio Showdown on the Dark Passage map, bringing out the heavyhitting Ragnarok Frank. That's a high-risisk brawler whose super can win the game. But the map's bushes can make it tricky to land. And we're in. Panda YT's team spawns on the left with a cult and a TIG, which so here basically just from a 10-second sample, the model is able to infer then from the script that it should have this very commentator style voice. Voice cloning can be used for this kind of entertainment. It can be also used for a lot of applications. So for example, now we are pouring uh a new application around restoring the voice of people with ALS. So disease that uh make people unable to speak in particular when people have access to recordings before the disease. Uh we can reconstruct their voice with exceptional fidelity. So in that context what you're going to listen to is Olivia, a French entrepreneur. And the nice thing is that he gave a lots of a lot of podcast before he got sick. And so we had access to a lot of data and we were able to to replicate his voice in a very natural way. And uh now you can also do uh personalized videos. So for example that's uh just a fun experience we are putting out there for Valentine's Day tomorrow. So here the main idea is that uh you can make uh like you can choose the partner's name. So I go with my co-ounder Lauron for the for the demo. And uh let me just check that I have the right microphone actually. And so I will record a small uh a short uh uh recording with my voice. So Lauron, this message is for you for helping me preparing all these live demos. Uh so thanks a lot for your support and uh really looking forward to do more risky demos later during this talk. So now I have my audio. So Lauron, this message is for you helping me preparing. So you can hear my accent and all the terrible aspect of my voice. And so now we're going to start generating uh with a photo like a video for that. We'll get back to it later and we'll mute this uh this music for the sake of the of the presentation.

So what we developed and open source so you can just uh use it with this single comment and is uh pocket TTS. So it's a 100 million parameter model running on a CPU. So not only it's its own device but it's running on CPU. It's competitive with a chatterbox uh and much bigger open source models. It's comparable to Koko that is quite famous but also comes with voice cloning. This uh CPU model allows you to generate highfidelity audio with voice cloning. This is a live demo of a CPUbased textto-spech model that delivers highfidelity voice cloning while running on edge devices opening up new opportunities for content creation. And so the fact that it runs on CPU now is allows for a completely new set of applications. So for example, there is already a Nexus mode life for awards legacy uh that allows people to have interactive discussion with NPCs that are purely generative. So here the LLM is still APIs based because we are far from having LLMs running on CPU but at least the generative part runs locally on the CPU.

The poor house elf. Did you help him? I'm afraid by the time I stopped laughing the elf had already operated away, leaving the boots to bury nothing but air. Don't look at me like that. I'm sure he's had worse days dealing with some of my more demanding housemates.

So basically now we have text LMS and we just want to interact with them through voice and the main idea is just to provide them with uh voice abilities. A very basic one is just uh voice of LM let's say. So you have an LLM and you add streaming speech to text and streaming text to speech. To give you an idea of how it works, let's see with the this small microphone. The main idea is that you can have real-time transcription and so as I'm speaking it's going to transcribe and this happens in real time and so I can give it that uh to the LLM and uh at the same time on the other side I will have uh the um streaming uh text to speech for the streaming text to speech at the same you're going to see the words being highlighted one after the other and it generated on the fly by consuming the text word by word.

Welcome to Gradium. If I could fix your problem by ordering you a pizza, I absolutely would. So here, the fact that it consum consumes word by word, it means that as your LLM is generating the text answer, you're already generating speed. So all of this is going to save you latency. Now an LLM, what you would have seen if it had worked is just a basic LLM like with just a prompt and it just say stuff, but it completely hallucinates. What is more important is to have an agent uh that is going to have access to function calling uh to different tools. In particular, we have created this opensource framework that we will open source soon called Radbot. Basically, the main idea is that now you can power agents with voice and when they have tools that can take time instead of waiting in silence for them to get the result, it just keeps the conversation uh going and when they get the result they inject it back in the flow of the conversation. So, let's try this one. Hi there, I'm Sophie from Wonderust Travel. How can I help you plan your next getaway? If you're not sure where to go, I'd love to suggest Paris, Bali, Dubai, Tokyo, Santorini, or Marrakesh. Each has something magical right now. Okay. Uh, I guess I want to go to Tokyo. Tokyo is an amazing choice. Right now, cherry blossoms are about to bloom. Imagine strolling through Shinjuku Guuan under pink petals. You've got two great options in Tokyo. The luxurious five-star Hikari Grand Tokyo with skyline views and a sky go or the charming three if you're excited. Got it. You're leaning toward the Perfect. Just let me know if you'd like to explore one of the hotels I mentioned or if you'd like to switch cities. I'm here to help. I'll go with the hotel in Hakusa. You mean Sakura House Asakusa? Great choice for that authentic cozy vibe near Senzo Gi Temple. Let me pull up the room details for you right now. And so basically, you know, this was vibe coded in a few minutes with uh with code. And so with Gradbot, what you can see is that when it's looking up information, it keeps doing this natural filler and then it can pop information. We are going to open source this uh this framework so that people can can try voice agents uh you know in real time pretty easily. Another thing we can do is live translation. So, one thing that we introduced uh a year ago is uh called Hibiki. It's the only real-time speech translation system and it can run locally on a on an iPhone. So, here what you're going to to see is real time uh translation with voice cloning running in airplane mode. Hi everyone. I'm delighted to be able to speak to you today. In France, my phone will translate what I am telling you to English. As you can see, it is an airplane mode. It doesn't have an internet connection because our machine translation model allows for real. So that was that we released last uh last year. In two days, we are releasing hibiki zero. So that's a new version. It's called zero because it's learned uh without aligning data. Now it's multilingual with better voice cloning. is that this voice cleaning also works for much less conventional types of of speaking.

Yes, it will enter the primary of Eloa. It is now in fourth place. Eloa passing there to cross. We are three laps from the final here in the Velo Romeo in Paris Olympic Games 2024. Portugal can So you don't hear it, but basically it's a Portuguese dubbing. It's like the the actual commentator is speaking in Portuguese. And here it's a real time uh the being done by our model. Win another medal in the Olympic Games, in track cycling, in the Madison race. There comes Portugal. There comes Portugal. There it comes. There.

And so now you can have even real time uh translation of sport commentator with all their speaking tiles and so on and so forth. I guess my video is uh is uh is ready and we can get a love message uh in video.

My dearest Lauron, I am profoundly grateful for your invaluable support in preparing our live demonstrations. I eagerly anticipate our future collaborations on even more daring presentations with deepest affection. Yeah, you could recognize the accent, I guess. So, you can uh get our API on Gradium.ai. We also have a lot of open source models uh on the GitHub of QI labs and you can already send a love message for free to your loved ones with bridger clone. Thanks a lot.