Um now, something here that I always see with these strides that we're making with robotics is A, these things are really, really expensive. They're all at least in mid five-figure range, so 50K up. Um and they don't seem to really be coming down. They seem to be targeting that price range for now. Um That's to say the humanoids. The Waymo, I think they're like six figures mid. Uh and they also they kind of look like something that you know. So, it's not very much about being creative. It's very much about trying to imitate reality. And this is not biology. This is really hardware, and we're really constraining the robots to do this. Uh if you take a humanoid robot, it could look like a spider uh and just move around way faster, be more agile, be more stable, but we're actually making it be a humanoid such we look at it and we think like, "Oh, yeah, robot, human, it's the same sort of. Like, I understand what it can do."

Uh that to me is a mistake. They also don't look very friendly in general. Uh so, I see a few problems with the current state of robotics. All of these robots, which they are making strides and they're becoming really, really good, they're still too expensive to prototype. I don't see any high school anywhere in the world ordering 10 of these 50K humanoid robots to let their students play with. Um they're very complex to adapt. They're really like targeting companies. And they You cannot really connect with them.

Uh and in the open source, we also have a lot of really good models. Uh you heard the talk from Mistral maybe if you were here before. Box Rally is a really good open source model. We also have tiny models like Cocoro at just 80 million parameters, a tongue good. We have good models to understand speech, and we have good pipelines like speech-to-speech from Hugging Face to put all together and make your voice agents yourself.

So, what I see here is that voice AI is quite mature. We have the tools. We have the tools commercially. We have the tools in the open source, but still, no one is really working on how we're going to talk with these robots. And we are thinking, "Okay, given that robots are coming, how can we manage to put this in the hands of everyone such that it's the experience of the future is not dominated by one company or one group of people, but really it's made like computers were made in a bit more of a hacker fashion."

Um a little bit of our vision. The idea is that it's an expressive robot that you can talk to it, but it doesn't look like a human. And that for us is very important because it already puts your mind in a different place creative-wise that you are going to start developing new ways of interacting with this, and it's not going to be just, "Okay, this is a human replacement." It's not. It's a new thing.

Uh and we want people to be able to explore that. Uh we made it affordable. We made it easy to use. Well, we're making it easier to use. We made it repairable. Actually, we ship these robots unassembled. So, the first thing you do when you get the robot is you need to assemble it yourself. And once you're done with it, which we get super positive feedback from, you can basically repair anything that's going to break in the robot ever. You have all of the knowledge. You have all of the tools. Um it really just is ordering the the parts and changing it.

It's very hackable. It's very cute. That's maybe a personal opinion, but I think it's cute. And we are trying to give you a set of software tools uh for you to develop with. Uh now, very quickly, we sell two of these robots. Uh they are both uh 450 and 300 USD. And the difference between both of them is the 450, which is this one that I have here, has a Raspberry Pi inside, and it has a battery. And that's it. Basically, if you don't need a battery, you don't need a Raspberry Pi, you can order the cheaper one. Uh we are selling those cheaper ones in bulk for high schools or universities to play with, to experiment to experiment, to hack with.

Um I here I wanted to show you a few um cool developments that we are seeing. For example, in the middle, what I mean with hackable, people can 3D print new parts for the robot. And in this case, they didn't need to really change anything. They're just taking the antennas out, putting a different type of antenna. They are putting something on top of the body. They are putting some lights, and they have suddenly a little Halloween pumpkin for to play with. Um on the right, something that we hadn't thought about, someone figured, "Oh, you can just pet the robot, and it can be fun, and it reacts." Does it purr? Um you can make it purr. Uh so, that's that's a little bit like for us, it's more about people exploring and people developing their own intuition for how this should be, and I'm making it their own.

...

And what we're trying to do is we're trying to give people the tools to actually do the conversation. So, everything that we are doing with the robot is open source. All of these models are open source. The agents are open source. Uh but we understand also that people are a little bit GPU poor maybe and they cannot run all of those models locally. We are also a little bit GPU poor and we have now shipped 7,500 of these robots, which means we have a pretty sizable fleet of people talking to the robot. It is our most used app by far. People just like putting it there and talking to it.

Uh and because you guys are a very technical audience, I wanted to show you a little bit how we're actually serving this. Uh there are three levels to the system. In the middle, there's the speech-to-speech pipeline. That's a project that I've been maintaining with Hugging Face for the last 2 years. Uh the Mistral Some from Mistral told you a little bit how it works. You have a voice activity detection system that knows if you're talking on or not. That sends it to a speech-to-text system. In our case, we are using Parakeet because it's super fast. So, we transcribe every 150 milliseconds and we send back the partial transcriptions for the robot to react if it hears something interesting. And when the transcription is is done, we send it to an LLM. The LLM replies, can also do tool calling for movements or use the camera. And then we send it to the text-to-speech system, which we're using Coqui TTS.

Now, on the higher level, we have the actual conversation app running in the robot. That is taking the input and the output from input from microphone and output through the speakers. It's doing the echo cancellation to not hear itself when it's talking and to try to hear you. It's doing the tool dispatching to move, do emotions, and it's using the camera. It can do face tracking to follow you around. Um that sort of thing.

And then the speech-to-speech pipeline, if you don't want to tweak it and use your own, we have it deployed in Hugging Face inference endpoints. We have a load balancer that determines how many compute nodes we have at at each time depending on how many robots are connected. And we have separated the LLM inference endpoints. We're actually using now Coqui 3.5 27B cuz we managed to make it fast enough. Um but we scale that differently because each of the conversation nodes can have an amount of concurrent users without the latency spiking up too much. And it changes a lot if in one node you have eight users that are talking a lot and using a lot the LLM. And in another node you have eight users that are not talking at all. Uh so, it really helps to save on resources if you have the LLM separated.

So, this is a model from Coqui that came out two or three months ago. For us, it was really uh a really great moment because TTS models hadn't been as good in the open source and as fast as Coqui 3 TTS when it came out. And we were really expecting for something like that. We knew it was coming, but we didn't know when it was going to come. Uh the issue is that the model that they released actually ended up being of the quality that they showed you, but not really of the speed that they showed you. Uh the paper claimed very low latencies, but the model didn't really achieve them. So, I spent maybe 2 weeks with Coqui TTS trying to get the model trained by them to actually be fast enough for voice agents.

Um and I wanted to show you a little bit a little bit how I did that and what the main issues were. Uh first issue that I found is that the model would generate the whole output before giving it to you. So, it wasn't streaming. Uh which meant if you wanted 10 seconds of audio, it needed to generate the 10 seconds. That can be solved with streaming. Uh it's harder in practice than in theory as things usually are, but when it works, it works and it's really cool.

The next thing that I noticed was the model being an autoregressive model was doing 500 steps for each packet of audio that it was generating. And for each of those 500 steps, it needs to coordinate the CPU with the GPU. So, it needs to send data back and forth between the CPU and the GPU. That is pretty bad. The way to solve that is to compile your model so that all of those interactions happen directly in the GPU. That couldn't happen by default because it was using a dynamic KV cache. So, the KV cache was evolving depending with the size of the inputs that it was processing. We changed that for a static KV cache. We used more RAM from the get-go, but that makes it faster. And then we could use CUDA graph captures to capture the whole model and to be able to accelerate generation significantly. Uh we went on a real-time factor from being below real time, so at 0.8, which means you generate 1 second, you take 1.2 seconds, uh to being 5.8, which means for 1 second, you take 20 seconds, I guess. Uh no, for 1 second, you take 200 milliseconds. Yes.