[Host]: Welcome back to the ACC podcast. Uh today we have Ali Masa the head of AI at Kavak. We're going to discuss today the transformation that Ali led within Kavak to turn it into an AI native company. Thank you Ali for being with us today.

[Ali Masa]: Thanks for having me.

[Host]: Before starting at Kavak uh you were running a company called Oppy Analytics.

[Ali Masa]: That's right.

[Host]: And you were very much into AI before ChatGPT. You want to tell us a little bit about that journey?

[Ali Masa]: Yes. Yes, of course. Well, we called it machine learning back then. It was a different family of algorithms and we founded a company with this very ambitious vision there that new machine learning models would be so powerful that they could solve any complex problem. This was pre-transformers, right? This was like 2013. So, we started building the company that way and I think we were like 10 years ahead of time. Uh but we built a great company. We served like Fortune 500 companies uh around like risk algorithms, logistics, forecasting, marketing and but like really the power of what Transformers and then like the ChatGPT moment uh when it arrived make things like very clearly that we could now build a whole new uh company and way of building companies and we joined Kavak to and Carlos to build that.

[Host]: Amazing. All right. So, we're going to spend the bulk of this podcast talking about exactly how you've identified Kavak. But maybe just to start, what does Kavak do and what is your role there?

[Ali Masa]: Kavak started out as a use case as a used car marketplace. So, we buy cars, we refurbish them, and then we sell them and finance them. But to do that, we also had to build a fintech and a logistics company and the Carfax and like basically all the infrastructure for this to work didn't exist in Latam. So we had to build everything vertically so we could serve our customers the right way.

[Host]: I'm going to sort of start with the framing of what the architecture looks like. So a consumer comes in and says I want to sell my car like how many agents do they touch? Like what's the harness look like? Like ground us in how you design this.

[Ali Masa]: Right. So, so, so we bet the company in transforming to a company run by agents. The questions we ask ourselves is how would we build Kavak in 2035 with Fable 10 or GPT 10 level intelligence and actually that company looks very different than what we had built or what we had back then. So when a customer comes in right now um agent will get spawned specifically for this customer with its own virtual machine. It'll remember years of interaction of this customers with Kavak what they visited in the web page or a call they had two years ago. Remember everything like in its memory. Come up with a strategy and set a long-term goal to maximize the lifetime value of this customer and do whatever it takes to make the customer happy and convert them into like all our different products and like across time and this is a completely new and groundbreaking architecture at scale. Uh I think because like people are still building multi-agent system with with experts and and we realized to bet that long-running agents with hard goals not just workflows uh could could maximize our our customers uh satisfaction and obviously their their lifetime value.

[Host]: Awesome. Okay. So we're going to jump to the nuances that but maybe versus many companies that say hey we want to be agentic and they try some workflows you guys took the just rip like we had to make this work you had to downsize dramatically it didn't work for a year

[Ali Masa]: right

[Host]: so do you want to talk through obviously you had to tune a lot of things to make that work like describe the harness at that time and like what models you were using and sort of specifically yeah

[Ali Masa]: So so there there were like three main decisions that that we had to make. The first and this is where I think many companies are stuck right now is the first instinct is okay let's adopt AI and you you basically leave your structure as it is and just give ChatGPT to your team and then there's no efficiencies your customers have the same problems and nothing happens right and so so you need to redesign your whole company around the agents and around the future capabilities and this means really like rebuilding most of your APIs, rebuilding your system so the agents can use them to to perform.

[Ali Masa]: Then you need to start generating the data and the feedback loops to fine-tune these agents. The only way to really make them work is if you teach them. And how do you teach them? You put them out in the open. You put them in front of customers. You get that data. You get those evals and then you train your agents. And this is the second bet that we made that we could build superhuman agents. This means that by every dimension that matters like conversion, lifetime value, uh customer experience, our agents would outperform the best human we had ever hired and we put them in front of the hardest problems.

[Ali Masa]: Um and finally you start to change how you measure the success of the company. Kavak was a transactional company. We used to measure how many cars we bought, how many cars we sold, how many brake pads we needed to buy. And we moved to a relational company where now I have 10 million customers in my database and I have agents assigned to most of them with the task of maximizing their lifetime value. Now, we're selling cars and personal loans and very high ticket items. So just activating 1% of this customer base, it's like hundreds of millions of dollars uh if we do it the right way. So it's a bet that made sense for us because of our industry, because of the ticket, and because at the end of the day, customers need to build trust with a company because they're buying a used car. And the way to build trust is to know them and to plan and nurture a long-term relationship.

[Host]: Ali, I just wanted to double click on something. You know, evals over agent demos. Yeah. Um you probably get pitched a lot of new agents and you know, it's never been easier to build things like before. But um one of the questions is like how do you guys go about evaluating this? Because not everybody test them across 90% of the customer interactions to see if they're really working. And you know you guys I believe is it about 98% of the interactions or something yes like that are now handled by agents.

[Ali Masa]: Yes. Totally. So to give you a sense of the scale: like 96% of all interactions uh are handled by agents. So no humans there. Um like 95% of all transactions are completely handled by agents. Obviously, you meet a human when you pick up your car, like there's someone physically there to give you the keys. But the rest of the experience of the journey is handled by an agent. Every day between 100 and 200,000 agents get instantiated in a day. They wake up, they work sometimes for three minutes, sometimes for eight hours, sometimes for three days, and they set an alarm clock for their next task and they go back to sleep. So the scale of this is just amazing and it's working. Now, how do you get this to work at scale? And the answer you mentioned it is evals.

[Ali Masa]: I like to move extremely fast, but in order to move fast, you need to have brakes, right? Imagine a car. You'll hit on the gas just if you have the right brakes. And AI is super powerful. And I've seen many companies get this wrong because they try to go slow because they don't have the right brakes. So I thought about it the other way around, like how fast can we go? Well, it depends on the quality of our evals. So, a good rule of thumb here is we spend about the same amount of time, engineer time, tokens, and money on building the evals than building the agents. And this is how you get better and better and better, not letting evals as an afterthought.

[Ali Masa]: So, what do we measure first and foremost? Like the results for the business. Like if my customer is happy, they'll buy a car, they'll get their loan approved. Uh they'll sell a car to us and that's the first check like did it convert? And that's where most things break. Like I see companies like measuring number of calls or minutes during the call or some superficial KPIs that give you some information but that doesn't really work. Like the important thing is did this customer convert? Is it bringing value to the customer and is the customer happy to re-engage with us after a while? And once you get those evals connected, then it's just optimizing the right agentic architecture and giving the agent skills to scale this and cater to millions of customers.

[Host]: It's really really amazing and you know related to this is like okay so you create the right evals it's working you know some people some companies still feel a little bit risk averse and putting them in front of the customers and being able to perform the highest leverage task which in your case would be selling. Do do your agents really sell to customers?

[Ali Masa]: Yes. So we never built customer support or customer service agents. We built like sales agents. It's extremely hard to sell a car in Latin America. So imagine someone wanting to buy a car. They can choose amongst like 20,000 SKUs. Then they need to pick like financing and go through the financing process, insurance and coverage. And then they're probably trading in their car. So we need to quote that car. So, it's a process that if someone does it or the way Kavak did it back in 2020 2021 was you need to be extremely good at 15 different things and have 15 different experts in 15 different teams. And usually the person would go and speak with the expert in financing, the expert in car advisory, the expert in buying, the expert in insurance, and they'll build a package and buy a car. That's extremely hard to do. But like the first thing we did was okay, can we get an agent to be better than the expert in each of these things and then put it together and have like a mega expert that's an expert in insurance, financing, etc. And that's who we put in front of the customer. So the experience for the customer is amazing. We tripled uh NPS and customer satisfaction score by putting the agent in front of the customer. And it at first it converted like 50% more than our human team and now it's converting over that like 2.1 uh x more. So it's a completely different concept.

[Host]: Your agents are better sellers.

[Ali Masa]: Totally better. And you get this right because they're experts and they're infinitely patient and they know all your history and they can plan for the long term and they never get tired. So, and if they make a mistake, they learn it and the next day, not just them, but the other 200,000 agents will have learned from that mistake. So, that's the feedback loop that we engaged and that's showing in the growth and results and satisfaction of our customers.

[Host]: Yeah. One of the well, one of two of the very cool things I think about Kavak is I think the world has gotten comfortable with AI can do customer service. It's still very hard to do well. But you know, as Gabe said, there's still a view that well, customers aren't going to want to buy expensive things from AI, and you are proving them wrong.

[Host]: The next layer on that is, well, you're not actually going to be able to do regulated financial services end to end with AI.

[Host]: But if you walk through what you're doing, you are underwriting a thin or no file customer.

[Ali Masa]: Yes.

[Host]: Pricing them correctly, doing servicing. So, so maybe talk through how did you write the evals to get comfortable with that and then versus I don't know going to a bank branch or even a fintech sort of how how is that experience?

[Ali Masa]: Yes. So much better.

[Ali Masa]: So the first financial product that we launched was a car loan and usually in Mexico and in some emerging markets it'll get like two months or more to get a car loan approved. Um, we usually approve it in under three minutes. Uh, which is like pretty cool because we have all this data around the customer and the car. And if the customer can't pay for the car anymore, they'll just return it to us and we can give them a cheaper car and then they pay a smaller amount each month and they like get out of the water, which is amazing about the vertical integration of the business.

[Ali Masa]: But then like when we started launching other financial products, we realized that this is a very important decision for the customer, right? Like they usually take three to four months to make up their mind and buying a car and getting a loan or getting a personal loan like a large personal loan that we also um do. So if you get to know your customer throughout this process and make the process easy for them, then just your conversion and retention metrics start going through the roof. It's not just the transaction is understanding each customer personally and get them to convert when they're ready with a very deep personalization of the interest rate, the risk, the max amount of the loan. In a way that makes sense for the portfolio as a whole obviously but that's optimized to the risk level and probably the other offers that the customer is getting.

[Host]: And then maybe give us just to be um you know evals are always a very hot topic. You kind of led with that. What is a like what is an example of maybe a hard to design area for evals or one where you had to spend extra amount of time with just given the fact that like there's real money PII at risk.

[Ali Masa]: Yeah. So when we decided to redesign the company around AI, you ask the question, okay, is AI going to be able to do this job like even the CEO job or jobs where the leadership is? And the answer honestly is probably yes. Like in 2035 with a rate of improvement, it will be able to do. So we said, okay, let's try it now. Let's try and build an AI CEO. So we carved out a city in Mexico. It's Guavaka. And we put like an agent in one of our harnesses as a CEO and it starts learning and it starts making decisions and evaluating on those decisions and it's only been running for six weeks now.

[Host]: The goal of the first month was to double the profits of Guavaka.

[Ali Masa]: It didn't reach it but it was 1.5x like 50% more profits. Just like managing the city which is crazy right it's amazing and it's the CEO like people were like that was the last job AI was supposed to take and no it isn't really. And how did this happen and it's like uh very smart person like Fields Medal level smart like going into every single number every single customer making the perfect forecast and going to micromanage every single things that needs to be executed every day to reach a plan. So he'll literally send messages to all the physical workers in Guavaka with their plans for the day and ask them to send voice notes back to know their progress. So customer satisfaction grew. Uh we got a better inventory. We rotated better, better financing penetration. Like every KPI started to improve. So, it's super cool. It's super exciting.

[Ali Masa]: Now, what are the jobs where we think uh we're still like training and hiring humans? Those are related to the physical world.

[Ali Masa]: So, when we talk about mechanics, Kavak has around I think in Mexico around 800 uh mechanics. There's lots of dexterity and senses that's super hard to substitute. So there we also build these agents with the exact same harness that's scaling and the mechanics have the sidekick. Um I was telling you guys earlier it's like the movie Ratatouille like the mouse that's actually a chef collaborating with a human. It's kind of like that. So it's a sidekick. We call it El Mike. And it tells them how to inspect a car and gives them tips and shows them the way to do it. And the quality of inspections again went through the roof. We're inspecting faster. We're repairing faster. It's cheaper. But most importantly, we're delivering higher quality cars. Um warranties came down around like 20 26% since we launched and customer satisfaction again went up. So it's about this like how would you design your organization from scratch? Uh with abundant super intelligence that's cheap and just go build it.

[Host]: Now, this is a good segue to a key topic right now in Silicon Valley where you know there's a lot of people worried about how the organizations of the future are going to look like um and the role that humans are going to play. Yes. In this and I think you touched a little bit on that. So would love to hear yeah like how you guys are thinking about that.

[Ali Masa]: Yes. And the organizations. Yeah, totally. So, um we took that question very seriously three years ago and the truth is that everyone's job will change. So, and what we were doing a couple of years ago will probably be performed better by an AI agent, right? So, what does this mean? We need to train everyone. So, we launched a program inside Kavak that's called the Jedi Academy where anyone from Kavak like from the CEO to mechanics.

[Host]: to Yeah. And it's awesome like from the CEO to like AI engineers to mechanics like going to the academy. It's super hard like I I've led

[Ali Masa]: I led them myself.

[Host]: You designed the program.

[Ali Masa]: I designed the program.

[Host]: but constantly

[Ali Masa]: constantly because you need to be upgrading the program because everything's changing so fast and there's like you can't send these people like outside to Stanford to learn this because like it's new stuff right so we train everyone and after six weeks they launch state-of-the-art AI agents to production and it's mechanics and finance guys and engineers is like everyone can do it and what this generated is maybe this person won't become an AI engineer some of them have but they know how to collaborate with this new technology right so the way we looked about it was guys there's no way back like this is the way Kavak is going this is the way the company will look like these are the changes for the engineering team the finance team the product team like this is what's going to change. You have the choice to like train and get the skills to perform in this new reality, in this new world. Um, or maybe leave Kavak if this is not for you, but this is the way we're going.

[Ali Masa]: And it was great like we strengthened the culture. Everyone was super excited. Um, people really know how to build this agentic systems. And then if you look at Kavak now any process it's really a collaboration of agents and humans and sometimes like agents are the bosses of humans and sometimes humans are designing the agents but I think we managed to really build this and change this and it's through this idea that we need to be learning every day and things will continue to change and the only way to continue being relevant is to upgrade your skills uh every month or every couple of months.

[Host]: But you do have or did have you know thousands of people now agents do most things. So like what is the org structure of Kavak? Like does the middle management concept even exist anymore? Like what does your org look like?

[Ali Masa]: Right? So the way it looks like now is very flat teams, very senior teams, super empowered. If you look at a team, you'll have engineering, AI, like operations, like everything. And they're either building the agents, working for the agents, or being in the physical world in front of the customer. Like most of our organization looks like that. So, so it's really built around the idea of how organizations will look like in the future and around AI and really harnessing this new technology. Obviously, this required lots of retraining because in 2023 or 2022, no one was building agents, no one was helping agents or taking orders from agents. And the way you cater to the physical world or the customers was in a different way than if an agent's telling you what to do or helping you make your job uh better. So it's a completely different structure than we had just two years ago.

[Host]: Yeah. Explain um we talked about this before what working for the agents look like. Like I think the way you described it was you know agentic system and then sometimes when it fails it's like oh that's kicked out to kind of a human queue right but then that's lost. And so how have you brought that together?

[Ali Masa]: So like the we see human in the loops and most of these agentic systems in production right now like large scale agentic systems usually if an agent hits a wall or can't perform anymore it'll like send this case or this customer to a tier 2 support and forget about it. That doesn't really work because you don't close the loops. So you don't generate the data to train the agent to do this better. What works right now is we have an agent that's obsessed with each of the customers like millions of this. They have access to every single API, every single skill like and we have humans building those skills for them. And then if an agent hits a wall or cancels something, it'll call this API saying I need help. And on the other side, it's not an agent or software, it's a human helping them out. But if you map this out in an org chart, it's really human teams that have an agent. I'm getting better results. It's super clear like it makes sense.

[Host]: That's actually a perfect segue. I know you get lots of leaders at larger institutions inbounding to you. So maybe this will save you many phone calls, but I think rationally many leaders of companies intuitively understand this. It is very hard still to deploy AI through their organization. Like the models are good enough you know that it's an org problem it's a psychology problem like what advice do you have or what have you seen?

[Ali Masa]: I think it's two things uh the first is it has to be top down because of this like if you just get adoption it won't go anywhere because it's hard to generate this taste or strategy for people to bottom up decide what to build and what not and come up with something that works for the company. So the transformation has to be top down and leaders need to adopt and leaders have to have a very clear plan on what to build. I've seen so many companies it's just like oh like we're doing a hackathon people are coming up with use cases. We're sponsoring some of these use cases. That doesn't work. It's like be very clear on what the company will look like in three or five years and then start building that and be like very vertical in guiding your troops towards that. Like an army doesn't really work if everyone comes up with ideas on the strategy and tactics and goes to the battlefield and like does whatever they want. Like you need a very clear strategy and that's what we need now. It's a like transformation stage.

[Ali Masa]: The second one is you need to measure what really matters and it's evals but it's also the right evals. So I see a lot of companies spending now huge amounts and they say okay I got adoption I'm just spending like hundreds of millions of dollars in tokens now. What about that like there's quality in the tokens. So have a framework here that's also useful like tier three tokens the most valuable are this agents where you can get the ROI of each specific token and I can do that now. That's great news for me because I'm growing and because I know the ROI of each token because it goes to agents that are performing the job of the organization. Right? These are the best tokens. Yeah, tier two tokens are things that you can measure indirectly. Do I see devs in the codebase and I can evaluate the value of these tokens at least indirectly and then push those to productions. Tier one when most companies are is people are just using plug code or ChatGPT or Copilot or whatever. What happens with those? I have no idea. So it's not just about adoption. It's really about having a very clear vision and then measuring that each token you spend is bringing you those benefits and just iterate iterate iterate from there.

[Host]: And I want to uh and we touched on this a little bit, but I think it's worth a dive as um maybe the most ambitious companies listening to this will decide to follow suit, which is you decided to build an agent per customer versus per task and then discovered along the way that each one of those agents needs its own micro virtual machine. So maybe kind of walk us through those decisions that architecture.

[Ali Masa]: Yes. And I think we're seeing these results now but it was a really risky bet because people usually go from workflows. Like if I could advise everyone don't build agentic workflows to graphs or functions or objectives and we built that that these are multi-agent systems that can perform a whole function for a complex goals like the ones I told that to sell a car you need to do financing, purchasing like recommendations, uh, etc. And we had thousands like tens of thousands of these agents working at scale running the business back in December. But then Opus 4.5 came out and I realized like this isn't the right paradigm anymore. Like the intelligence now doesn't need like the graph and the multi-agent lattis work and harness because it will constraint this level of intelligence. So we decided to like destroy everything we had been building for two years that was working that brought us to profitability that brought us amazing growth and start over with a harness that we thought would be robust and scalable and leverage recursive self-improvement or new models more intelligent models coming out every month.

[Ali Masa]: So the way this looks like it's a virtual machine with an agent with access to memory and evals and the CLI where they can access every tool and every API in my company and the long-term goal and I instantiate hundreds of thousands of these each day with long-term goals like maximizing the lifetime value.

[Host]: Yeah.

[Ali Masa]: The self-improving organization.

[Ali Masa]: Exactly. The self-improving organization. And I think people are super obsessed with RSI now and this will improve the models. But if you look at it this way, economic value in humanity for the past 4,000 years has been delivered by organization, not by individuals. So what you want to self-improve and to engage in that loop is the organization that can deliver more economic value. Right? So that's the loop that I think companies will start to focus on because if you get that loop working and it's an organization that is really self-improving and harnessing the newer models and the better intelligence that we're getting every couple of days now uh then like you hit the exponential not just in intelligence but in the value uh that you can generate as a company. So so that's really exciting. That's what we're working on.

[Host]: You mentioned that because of all the challenges on adopting AI, you saw the biggest opportunity on net new companies being formed working on this new way and then disrupting markets like you want to talk a little bit about that?

[Ali Masa]: Yes. Um there's this concept in economics about creative destruction from Joseph Schumpeter and what it says is that the way innovation hits the economy isn't by companies adopting the new technology but by companies remaining the way they were and incumbents with the new technology destroying the old companies. So this destroys value in the short term in the economy but in the long term it's better for everyone because this new more efficient more effective companies will provide better products and services for the economy as a whole and this has happened in the past like industrial revolutions and this has always happened and this a great opportunity for entrepreneurs and people today because it's hard to adopt AI deeply. It's really hard for a CEO today, especially of a large company or public company, to go and say, "Hey, like I'm betting everything on AI. The company has to look this way. I'll destroy and rebuild everything I've been building for the past 40 years to become an AI native company." Like how many CEOs will do that in a company at scale. So while they adopt, new companies can be formed that are built around the strengths of AI and take over and bring new products and services to the masses.

[Ali Masa]: And um this has happened before like this happened with electricity. This is a story I always tell my team. The technologies for Ford's production line were developed in 1879 and 1881. Edison started commercializing electricity in New York and then London and he invented a dynamo that was extremely efficient. So you could have built Ford's factory 40 years before Ford. The technology was there, everything was there. But the way people adopted electricity and Ford's dynamo was okay, I'm gonna leave my factory like four floors, shafts and belts and just change my coal engine for an electric engine. And this will bring you benefits, yes, but like 6% efficiency. What needed to be done was like to destroy that factory, build it in a flat surface, not in the center of New York, but in Connecticut or New Jersey, and redesign your whole factory around small dynamos and electricity. And then you get like the 3x improvement in productivity that powered the US during the 20th century. And the same happened again with a computer. And the same is happening again today. People want to adopt it, but they're not willing to redesign the whole company and they just adopt it superficially. And in the end, that'll give you a 6% or a 10% improvement, not a 10x improvement. And it's like the innovator's dilemma at an industrial scale. Again.

[Host]: I think you've just made an amazing case for any future founders out there that it's time to build.

[Ali Masa]: It's time to build. And maybe a great place to end is, you know, you've built and scaled your own company, you've now turned Kavak fully agentic, like what advice do you have to future founders or first-time founders that might be listening?

[Ali Masa]: So, this is the most exciting time in human history. I believe that we're living in the most exciting time in human history and it's the most exciting time to be a founder because it's the first time that anyone has access to the most powerful tools and intelligence in the world like for almost for free or for $20 a month. So, literally the democratization of the tools for people to build has never been this way in human history and there's so much problems to be solved and a new reality to be built around this new paradigm. So say like just go for it but go for it deep like imagine what the future around AI will look like.

[Host]: It's just a uh it's not even an exponential. Just map a trend that's linear. If things keeps getting like AI keeps getting better at a linear scale and just build for that and you'll come up with wonderful ideas that will like bring a lot of value to the world.

[Host]: Amazing. Ali, thank you for joining us.

[Ali Masa]: Thanks for having me.