Sarah Chang: Hi everyone, we're about to start the next session. Thank you guys so much for coming out today. Um, this is going to be a build your own sales agent workshop. So, we're going to be walking through everything you need to know to build your own voice agent. My name is Sarah Chang from Cerebras and I am excited to be joined by Genway. Um, and we are both part of the DevX team at Cerebras.

Genway: Yeah, thanks Sarah. Um, so today we're going to walk through how to build a voice sales agent that can actually have a natural conversations with customers and our sales agents will pull product contacts from an external source to respond in real time. So, we're going to be building an AI agent that can speak, listen, and respond intelligently um to your company's sales materials. And we have the full code for you to follow along with. We have a notebook that you can scan later um to step through and we'll walk you through it step by step in just a moment.

Sarah Chang: So, before we get started, let's go through what you will get out of this workshop. So you will get free API credits for Cerebras livekit cartisia. You will have the quick start. We'll have again have a full code notebook for you to follow along with and at the end you will have your very own sales agent that you can hook up to your company's materials so that you can you know implement this in production. So here's the starter code that I would recommend scanning just so you can follow along. Um, again, this is what we'll be walking through step by step today. And there will be individual modules that you'll be able to just run and see some good outfits. So, I'll give you a few seconds for that.

Sarah Chang: We'll have the QR code later as well, so not to worry. So, before we get started, I wanted to talk a little bit about Cerebras and, you know, Cerebras inferences secret sauce. So, for those of you who are unfamiliar, we are a hardware company. We are building an AI processor that is much larger and much faster than what you are probably familiar with with Nvidia GPUs. So out of curiosity, I'm wondering how many people here have heard about Cerebras hardware. Not bad. Okay. Higher than last year. Okay. Okay. So before we do go, I want to share um I want to show everyone [clears throat] the speed of what we're talking about here. So So this is just a chat. It's running on Cerebras. You can choose any. So, we can host any different model on our hardware. So, I'm going to choose an example model like a llama model. And I'm [snorts] going to give it a prompt. So, I'm going to give it a prompt that it's intentionally asking it to respond something a little longer. This go [clears throat] funny dad jokes, but make each joke a couple sentences. Sentences. And that's how fast it generates. Does anyone else have a prompt you want to try? A longer prompt.

Genway: Amazing. There you go.

Sarah Chang: So, really quickly before we get started, I know we have a lot of software geeks here, but I do want to for a second talk about hardware. And I want to talk a little bit about what hardware innovations um make such fast inference possible especially as we build a new generation of AI products. And so we're going to a little bit of a hardware segment, but one of the main secret sauces for Cerebras is that Cerebras chips do not have memory bandwidth issues. And I don't know how familiar you guys are with, you know, GPU architecture, but we're actually gonna de deep dive really quickly into how GPU architecture works and how it compares to what people are doing today. And so for context, this is the hardware that, you know, all of our inference runs on. It's the wafer scale engine 3. It is quite literally the size of a dinner plate. And this has 4 trillion transistors, 900,000 cores, and very significant amounts of onchip memory. And so this is the comparison of what our hardware looks like next to the NVIDIA GPU. So you can see some of those metrics line up. So significantly more transistors.

Genway: But to actually understand what Cerebras did with their hardware that is makes it able to achieve 20x 30x f 70x faster speeds than in inference on Nvidia GPUs. We're going to actually start by taking a look at the Nvidia GPU. So this is a diagram of an H100. And if you look at the red rectangle, that is a core. And so on the H100 there's about 17,000 cores and each of these cores is the is what is actually doing all of the mathematical computations needed in training or inference or whatever computation you need to do. So every core has a subset of the computations um that is assigned. So when you run inference what are some of the types of things that a core will need access to to do its computation? it needs its weight, activations, KV cache, etc. On the H100, all of these values are stored offchip. So, they're stored in an offchip memory. And so, as you can imagine, during inference, each of these cores, there's thousands of computations happening constantly. And each core is needing to constantly load and offload the KV cache, activation, weights, etc. from an off-memory location. And as you can imagine this creates a very significant memory channel um memory bandwidth bottleneck.

Genway: What Cerebras has done instead is that instead of storing all these values off chip every single core on the Cerebras hardware the WSC3 there's 900,000 cores which in comparison to 17,000 is already a lot larger. Um every single core has direct its own direct onchip memory. So its own SRAM. So every single core on this wafer has a memory right next to it. And what that means is that all of the values that every single core needs for computations like weights, KB cache, etc. is directly accessible and much faster to accessible and it's right there. And so as you the other and so that's a little bit that's one example of what Cerebras has done on the hardware side.

Sarah Chang: Um, but going back to software, I also want to talk about really quickly one thing that Cerebras implements on the software side to accelerate inference. And so one way that you can accelerate inference is through a technique called spec um standard decode or speculative decoding. So in standard decoding you have one model generate every single token one at a time. And this is sequential, right? You have to wait for the previous token to be generated to generate the next token. So in speculative decoding, you combine two models. And what you're doing is you use a smaller model that's like a draft model that can generate all of the tokens very quickly. And then you use your larger model to go back and verify that the output of the smaller model is correct. And by combining these two models, you're able to get the speed of the smaller model and the accuracy of the larger model. And if you think about it, your speed is capped by this uh your like this the speed um is capped by the speed of the larger model. So you will up to the large like the speed will be up to the larger model um but it will never go beyond it. So it will only be ever be faster.

Sarah Chang: So as a kind of a short recap, hardware, memory, bandwidth, we talked through that software, specular decoding, but that was a little side moment and I want to go and now back to the workshop. Now that you have all the context that you need.

Genway: Awesome job.

Genway: Yeah, thanks Sarah. Um, for those who folks who join in late, you guys can scan the QR code to get the starter code. We had it in the early slide, but um since we'll be teaching you guys how to build these sales agents, you can follow along with our code. Um yeah, so I think in the future, most customer interactions will probably be AI powered, but you know, instead of just typing back and forth with the chatbot, what the best way to kind of really have these customer interactions is really through real conversations, which is why voice agents are so powerful. So before we dive deep into it, what exactly is a voice agent?

Genway: Absolutely. Um so voice agents are stateful intelligent systems that can simultaneously run inference while constantly listening to you when you're speaking and they can actually engage in real and very natural conversations. Um I would like to highlight four key uh capabilities. First, they understand and respond to spoken language. um they don't just spit out answers based on string matching or keywords but rather they can actually understand the meaning behind what people are saying. Um this also means that they can handle a lot of complex tasks. So someone might ask like I'm looking for a product recommendation and the agent can subsequently kind of look into the users's purchase history, the shops's current stock levels and recommend something that they actually like. And you actually might see this referred in some places called multi-agent or workflows. Um speech is obviously the fastest way to communicate your intent in any system. We're speaking now I guess [laughter] but you can just say what you want. There's like no typing, no clicking through menus and no learning learning curves. And lastly um none of this would be possible unless the agent can keep track of the state of the conversation. uh which means the communications obviously is very highly contextual and your agents needs to have like state so they can actually hold a coherent conversation across time. So as you can imagine this makes um voice agents perfect. You see a lot of startups happening right now especially in customer service, sales, tech support etc. And so today we're going to be focusing on the sales agent use case. So, first let's talk about what's actually happening inside a voice agent when you're having a conversation and break it down.

Sarah Chang: Yeah. So, you guys can see on this diagram on the right, once speech is detected, the voice data is forwarded to ST or that's called speech to text. This listens and converts to your your words to text in real time. And the last step in this process is end of utterance um or end of turn detection. um being interrupted by AI every time you pause. It's like very annoying. So, while VAD can help the system know when you are and you aren't speaking, it's also very important to analyze like what you're saying, the context of your speech, and to predict like whether you've done sharing your thoughts. So, we have another small smaller model here that runs quickly on the CPU, which will instruct the system to wait if it predicts you're still speaking. So, once your turn is done, the final text transcription is forwarded to the next layer.

Sarah Chang: And then after that phase, we have the thinking phase. So your entire question is now passed onto the large language model. Um, and this is basically, you know, the brain like understands what you're asking. So it might need to look things up, which we'll walk through later. Um, like checking in this case, if we're doing a sales call, we'll want to pull additional context like documents, your other like more information about your company basically.

Genway: Yeah. And then the third and the final step is the speaking phase. So as LM streams response back to the agent, the agent will immediately starts forwarding these LLM tokens to the TTS engine or text to speech. Um this generated um audio from TTS streams back to your client's application in real time and that's why the agent can actually start responding when it's still thinking. So the final result is that all of these components tied together is what's making, you know, an AI agent that feels very responsive, that feels very cohesive and immediate, even though there's a lot of complex processing happening behind the scenes. So there's a lot of moving pieces. In this case, we're going to be using LiveKit's agent SDK to handle all this orchestration for us. Um, it's going to manage the audio streams, keep track of the context, and coordinate all these different AI services that we've just talked about.

Genway: So, now that we have a little bit of context, um you can access the starter code here. We shared it already. And if you want to run the first section right here, it'll allow you to install all of the necessary packages. So, if you click on it, um you'll be able to see some of the output of the packages being downloaded. And so, this is going to use live kit agents with support for Cartisia, Cilero for voice activity detection, and openai compatibility. And so we've very briefly talked about Cerebras. It is 50 times faster than GPUs. And um I'll skip here. And so as a final note, so for this um for this workshop, we're actually going to be using Llama 3.3. And if you see in the chart on the bottom right, this is a chart from artificial analysis. Artificial analysis, if you're unfamiliar, is an independent benchmark that benchmarks a lot of different models, API providers, etc. um on intelligence, speed, latency, everything. And so you can see a comparison here of Cerebras on the very left in terms of tokens per second and any of your other providers like Nvidia.

Sarah Chang: Awesome. Um going back to our code, um hopefully everyone has had a second to kind of install the packages. Um, and now let's also in we can also install the live CLI. This is optional for our workshop today, but if you want to use live kit beyond this, um, here are the commands depending on your system. Um, in general, we're obviously using Python notebook today. So, no one has to battle around your environment when we're getting started. But again, if you want to continuously build and deploy uh the voice agent, the CLI probably is the easy way easiest way to do it. So just uh type in LK app create and you can instantly clone a pre-built agent like this one.

Genway: Cool. And um let's talk a little bit about what exactly LifeKit is and why we need it for a voice agent. So the existing internet isn't exactly designed to build voice agent a uh application. So HTTP stands for hypertext transfer protocol. So it was designed for transferring text over a network and obviously for what we're building we need to transfer voice data instead of just text over a network with low latency. Um and kit is a real-time infrastructure platform for doing just that. So instead of using HTTP actually uses a different protocol called web RTC to transport voice data between your client application AI model with less than 100 millisecond of latency anywhere in the world which is awesome. It's very resilient, handles a lot of concurrent sessions and it's fully open source. So you can kind of dig into the code and you can see how it works or even host infrastructure yourself as well.

Genway: Um yeah, so you can use live kit to build any of type of like voice agents, the ones that can join your meetings, the ones you're answering phone calls and sell centers and call centers and in our case today an agent that can speak to prospective customers on your website on your behalf. And here you can see connecting it to the original diagram that we showed. So you see like the LLM, TTS, ST and all the AI components that we talked about earlier. And now you can see, you know, how these actual tools like Live Kit, Tart, Cartisia, your inference provider, all of these things are actually playing together to help you create a voice agent. And so the final component as I mentioned is the actual speech processing um which so in addition to cerebrus and lifkit and as I mentioned we'll be using cartisia to turn the voice into text and then at the end text back to voice.

Sarah Chang: So now that our API keys are set up, step two is all about teaching our AI sales agent about our business. So when you train a new employee, you have to give it information and context on your business. And so that's what we're going to be doing now.

Genway: Yeah. Um, I think the challenge a lot of the times with LLMs is that they know a lot about everything, but they might not know many specific things or domain things about your company. Um, and they're only really as good as their training set. So, if we want to respond with any information that isn't common public knowledge, we should really try and load it into the LLM's context to minimize hallucination or any sort of canned responses such as, "I can't help with that." So, in this case, we're just going to be feeding the LLM a document with additional information. So, for example, we can load our pricing details if someone asks about pricing. But we can also load information like product descriptions, pricing info, key um key benefits. And another big thing that we can do is write pre-written responses to common objections. So, for example, if it's common that someone says it's too expensive, you can write a pre-written message so that our agent will always stay on message and it has the correct context. So, if you look at the notebook, you can see what that context looks like in practice, right? you don't have to just give it access to a full document. Um you can see that we've in um organized all the information that our sales agent needs into a very simple structured format for the AI to understand and reference. So you can see everything that you um a good salesperson would need like the descriptions and then as we mentioned it has these pre-written messages as well so that you can control the out um the behavior of your voice agent more closely.

Sarah Chang: Um, now we're off to the more exciting part, even more exciting part, step three, where we actually create our sales agent. So, this is where everything that we've just talked about, the components, and we're going to wire them all together into a working system. Um, and before you run anything, let's actually walk through what is happening in the sales agent class. So, in the code, you can see we start by loading our contacts by using the load context function we defined earlier. And this gives us our agent access to all the product information, pricing, and objection handlers that we set up. Oh, sorry. So, and finally, I want to look at how we're implementing everything in code in terms of creating the actual sales agent. So the there's way more of the code in the notebook, but as a high level um you want to start there's kind of four components. So you want to start by you know telling your sales agent your voice agent communicating um your sales agent commun communicating by voice um and give it proper rules like you know don't use bullet points because everything is spoken aloud. So you want to do um a bit of prompting and then most importantly only use information from the context that you provided. So you want to make be very careful especially with voice agents that you are not allowing um that you're reducing the risk of hallucinations as much as possible. And then the super call is what's initializing our agent and passes all of our configurations to the parent agent. And this is setting up our agent with the LMC TTS VA and all the instructions working together. And then the last thing that we're going to do is we're also going to define an onenter method which is what's going to start the actual conversation. So, as soon as someone joins the conversation with the agent, instead of sitting in silence, it immediately um or this is triggered as soon as someone joins the conversation.

Genway: So, instead of ever sitting in silence, you're going to immediately generate that grading um and the good salesperson will help. Yeah. And then we're off to our step four. We're actually launching a sequence and running the agent. Um, think of this entire kind of uh entry point function as a start button to our agent. And when someone wants to have a conversation, obviously it kicks off every in the gear and gets the agent ready to talk. So this entry point function is doing three main things. So it's connecting the agent to a virtual room where the conversation will happen. So it's like dialing into a conference call. Um, then it's going to create an instance of our sales agent with the setup that we just configured. And so finally, it's going to start a session that manages the back and forth conversations. And so that is it for the basis or like I guess the main framework for how you would set up a sales agent.

Genway: But to make this project a little more robust, we're actually going to talk about one a few ways that you can expand your sales agent. So here's one example. Yeah. So one thing you can do um to expand our single agent into a multi-agent system is um to just you know if someone calls asking really deep technical questions about API integrations you really want them talking to your best technical person and not just your spicing pricing specialist. Um again all limbs have limited context windows which means that similar to people they have limits on the amount of things that they can actually specialize. Um and here are the three other agents in addition to that single agent that um the the starter co has just helped you guys run. Um three of the different agents that we propose in this case are um greeting agents um our main sales agent who qual qualifies leads. We have a technical specialist agent as you can see on the left um who are obviously specialized in sol solving technical issues is the intent and then finally we have the pricing specialist agent on the right which handles budget ROI and also deal negotiations. So the main thing that you want to think about here is you know on a real sales team you want or any like multi-agent system you want all of your agents to be able to do very different things. And so one of the key things in this um implementation is that we have a um is that we have a handoff. So our greeting agent is what figuring out what the customer actually needs and then being able to route to the um to the relevant sub agent. And the code for all of these different agents is fully fleshed out in the notebook as well.

Genway: And then the last thing of course is you can is adding tool calling. So for example when someone a customer asks about technical details you know we can properly route and then this is also implemented as well in the code notebook and that is it. So thank you guys so much for coming. Um all again all of the notebook with all the instructions and the step by step is in the notebook that we're provided and have built. Um and we'll be up here to answer any questions that you guys might have. Thank you guys.