专用模型为何能击败全能“神级模型”：OpenRouter 与 Replit 探讨多模型生态、模型对齐与企业智能独立性

Alex Atallah: I had talked to Will Gaybrick, like Stripe President, a long, long time ago, a couple of years ago, when we were doing our Series A. And we just stayed in touch. We held a lot of Stripe work streams going on with various teams at Stripe. There were always sort of like things that we were doing with Stripe. We presented at Stripe Sessions. And so they always kind of felt close. And then, yeah, just in July, I believe they reached out and wanted to chat. I met with both of them in person. And it kind of just progressed from there fairly quickly. They are very efficient, and they were very, very founder-friendly. The whole experience was really impressive.

Amjad Masad: Did you want to sell? Like, did it even cross your mind before they reached out?

Alex Atallah: No, we were not thinking about that at all. I did really respect the company, do really respect it today. And you know, of possible acquisition options for us, it was my top choice. So it was like an interesting idea. And, you know, as we kind of fleshed out the reasons why it would make sense for both companies, it got more and more interesting. It was really clear how aligned they were with us: having autonomy over the brand, roadmap, and product, and keeping OpenRouter doing what it's already doing, just much faster with a much more serious go-to-market plan. And some better-together stories between the two products and the two companies. Culturally, in terms of mission and values, building a neutral, trusted platform that businesses can depend on and scale on top of, that's also really developer-friendly with the best possible developer experience to encourage new companies to emerge. That alignment was there. And there was a bigger-picture alignment, too: both Stripe and OpenRouter really want lots of new companies in the world, and we don't want everyone to be a part of one giant company. We want to create really good incentives and really streamlined workflows for people to start new companies and grow them successfully, making both lifestyle and venture-backed businesses on top of really good, reliable, and price-efficient infrastructure, and like a marketplace that works. I really want that future, and Stripe demonstrated that they've been wanting it and building towards it for many, many years. In many ways, payments and inference are going to blend together for companies of the future.

Amjad Masad: I'm curious. I understand why Stripe's incentive is to have a much more vibrant startup ecosystem. I understand the moral argument in why you would want that. But why is that good for OpenRouter? Is your model for OpenRouter a network effects business?

Alex Atallah: I think there's a couple of different problems OpenRouter resolves. One is allowing you to build a company that uses AI or augments intelligence with unique data and other services without model lock-in, without venture lock-in, allowing you to be on the Pareto frontier continuously as the ecosystem grows. To do that is a lot of work because there's all kinds of little lock-in that appears. We also want companies to feel like the best way they can add value is more than just prompts on top of a single model; there's a lot more to building unique intelligence. I think a big component of that is neurodiversity. You really need the power of multiple models that are trained in different ways, including some of your own, to do more than ChatGPT or Claude would on the task. If a potential customer is wondering, "Well, what if I just use the model directly?", how do you really show that you are significantly better and able to build a business that matters? A lot of it will involve neurodiversity and blending powers and good data from multiple models. Another component is helping people get really good cost efficiency. There are a lot of businesses that just cannot emerge until they become cost-effective. Creating an environment where we can help drive down costs by building an efficient market is crucial to making that happen. Otherwise, why would a provider lower prices? We had a captive market. That's a key point of marketplaces that was totally missing for AI before we showed up—there was just one player, OpenAI. Helping people explore new models and learn what makes a closed-source or open-weight model actually good involves seeing what the whole ecosystem is doing. LLMs are not like where you just enumerate all the features on a web page; it's impossible. You have to see how they're being used to know what they're good at.

Host: So you started the company three years ago. I'm curious, what has surprised you the most about the evolution of open and closed-source models to the present, as relates to model performance or just people's perspective on the variety of models they have available to them?

Alex Atallah: People have been more open-minded than I thought they would be towards open-weight models. Typically, there's a lot of brand trust, especially in enterprise, where enterprises in general say, "Oh, I really don't know how to tell the difference between these things, so I'm just going to buy the one all the other credible enterprises are buying." There's a lot of enterprise lock-in mentality like that. That just didn't happen that much; we saw a lot of enterprises want to explore new models. It was very good for a marketplace. Enterprises wanted to diversify outside of just the proprietary frontier model labs, both for cost reasons and for differentiation reasons. They wanted to own their intelligence so they could keep their talent and have an internal AI practice. AI is just a huge strategy topic. It's not like you go to your board and say, "Oh yeah, we fixed the AI problem a quarter complete." Your board is asking you every month, "What's next for the internal AI team?" Every single enterprise now has this internal AI team they're developing where they need a strategy behind it. It's not just a "we checked the feature, set up the database, and we're done." That dynamic has resulted in a desire to explore and diversify, to figure out how to reduce costs and do benchmarks for the first time in the company's life. I have been surprised there hasn't been more benchmarking, but it's starting to happen, to demonstrate whether something is better than using Claude direct. That's going to be a bigger focus for this internal AI group at every company—evals. Amjad's been doing that a lot at Replit: context research, doom loop rescue, experimenting with new ways of using agents. More of that kind of research is going to pop up internally everywhere.

Amjad Masad: I think Satya, CEO of Microsoft, has been very prescient on this and very articulate on why companies need to own their intelligence ultimately. In the same way that we had dot-com companies and then every company became an internet company—every company employed people that know how to build websites—similarly with software, every company needs software engineers, and every company needs some AI practice, AI capability. That will compound over time: the knowledge, the intelligence inside the company, the use-case model fit—which models actually work for them, how do they save money. They need that independence. The other thing that Alex Karp of Palantir has been talking about is the risk that when you work closely with the foundation model companies, they're going to move into your business. We've seen that with Anthropic, we've seen that now with Harvey and OpenAI. It's really hard to partner with them because they see the world as their potential market, right? When they talk to investors, they talk about space as tens of trillions of dollars; what is the world GDP, a hundred trillion? There is a sense in which these companies are different than other generations of companies: it's harder to partner with them because their ambition is such that they want to subsume a big part of the economy. Increasingly, what we're thinking about at Replit is becoming more of an independent layer inside enterprises where we create a light layer of indirection between you and the models, getting you the best token at the cheapest price, but also creating an abstraction layer on top of the cloud as well. You should be able to deploy to AWS, Azure, and use Databricks, Snowflake, and so on. There needs to be more platforms that help companies gain independence.

Alex Atallah: This reminds me of this tweet I saw the other day where somebody was like, "Basically all companies are building the same thing now: everybody's building an agent loop with notifications, third-party connectors, context management, memory, sandboxes, web search, and an always-on agent on top of it." It's like this product is showing up everywhere. In a way, yeah, it is showing up everywhere, but it also feels to me like these are just the new table stakes primitives. It's kind of like a 2005 version of that tweet would be: "Oh, everybody's building the same thing: a database, a users table, a sign-in page, a sign-up page, a profile page, a log-out page." There are table stakes needs for AI, just like there were table stakes needs for the web.

Amjad Masad: Yeah, and I think as well, inside the enterprise, making these products actually do real work is still an unsolved problem. You can use them in your personal life and connect them to your credit card and bank accounts, but no one is connecting this to their enterprise data or even sensitive customer records. There's an even more emphasis on data sovereignty and security. We spent the past year working on making Replit deployable on your own cloud—basically on-prem, bring your own cloud. Two years ago, I would have thought I would never do this because the cloud is the future, Software as a Service and all that. But now we've reverted a bit back to a world where companies are more protective because there are so many ways in which data can leak with all these agents that people are using. There's all these screenshots out there where assistants mix people's data, start calling you by a different name, or something like that. Consumer stuff is kind of obvious, but on the enterprise side, there's still a tremendous amount of work for the entire industry to do in order to actually get these things to be useful and productive at work.

Alex Atallah: Are you doing any custom personal agent other than general chatbots that you use for work stuff that you've been building?

Amjad Masad: Yeah, I built something on Replit a long time ago. It started as sort of a CRM agent initially; that was the main problem I had. But slowly we added features to it, and it started doing more and more things. What's really interesting is that the more I connected Replit to all my tools, it started answering all the things for me. The platform itself began subsuming these domain-specific applications that I built. In some ways, you want something focused on one particular thing, but on the other hand, once you have your entire company's context in one place, it's really cool to join across totally different domains. When I ask a question, it can look at my chat history, join it across the GitHub repo, across Salesforce, and link random things: "Oh, you met this guy a year ago at a conference, I see it on the calendar, and someone else from their team is in discussion with your sales team." It creates all these synergies. When I go into a meeting, I've connected a lot of different threads and make much more progress on a deal. That's where it's trending now.

Alex Atallah: I'll take the counter on that. I think the worst part about doing cross-domain joins with your personal agent is that the more work you give it to do, the more understanding of what's going on you're sacrificing. And yet, no one new is taking responsibility for that sacrificed understanding. Agents don't have any responsibility. If there's a fixed level of cortisol that the whole company can tolerate between everybody, and you want to be less stressed about some area: if you're sacrificing your understanding of it, someone else needs to take the cortisol, but the agent doesn't take on any of that responsibility. Giving universal agents the power to do all things makes me realize how much understanding I'm sacrificing across all different things. It points me towards the idea that down the road, sub-agents people use will be very vertically focused. Maybe we have a Chief of Staff type agent that coordinates between them, but you do need vertically focused agents: "Okay, this agent is more responsible psychologically for these things, and I want quality checks that make sure it's doing those things correctly, and it doesn't need to focus on anything else." Having one general agent creates a tragedy of the commons. Every time I try to improve my general agent, I end up ignoring its output a week later. It doesn't really care about any specific thing it's diving into. Imagine having ten Chiefs of Staff, each responsible for individual sectors of what makes up your life; that gives you a way of tuning how much understanding you sacrifice compared to the gain you get.

Amjad Masad: That's fascinating. It's almost rediscovering the famous economist Adam Smith, with the pencil and the division of labor. That was a huge realization for humanity: that specialization is actually good. The problem is we kind of over-specialized as a civilization, and there's the Marxist theory of alienation where people are just focused on one thing, they don't see the fruits of their labor, they don't know their impact on the larger organization, and therefore they feel depressed and detached, acting like a machine instead of being fully human. With AI agents, we think, "Oh, it should be one God agent." In fact, specialization is really good for machines. Humans should be general, but machines should ultimately be a lot more specialized.

Alex Atallah: The problem with what I'm describing is that we don't know what "good" looks like. There hasn't been a system of specialized agents that feels as elegant as ChatGPT or Claude, where you're basically just talking to one thing only; it is yet to be discovered. When Grok bot first came out, the first use cases I saw were: "Oh, I can make two bots, one that knows my bank account and one that knows my Twitter account. The two bots don't have credentials from each other, but they can talk to each other to get something done without credential sharing." That was one unique thing that popped up, but it's still very early.

Amjad Masad: Personal agents are different than work agents. The critique of general agents pertains more to work and enterprise, which I agree with. There are data access considerations: CEOs can have general agents because we have admin access, but individual employees or certain teams can't have truly general, fully context-aware agents because there's access control. You have to lean on specialization. Ultimately, we also need to figure out what agent communication protocols look like. There are no good protocols around that just yet, and agents are not trained to handle that very well. We've seen that OpenAI models are starting to collaborate, but there needs to be a way where an agent cannot convince another agent to give it information that it shouldn't have. There needs to be data isolation and proper ways in which these agents communicate. You almost don't want them to communicate fully in natural language; there needs to be some other DSL protocol that they follow.

Alex Atallah: I think one of the cool potential applications of fast decision models like that is going to be alignment: checking to see if a tool call or an agent-to-agent communication is aligned. There are just so many tool calls; you really need a cheap, fast model if you're going to block something like that. A really fast decision model that just classifies and gives feedback on rejections might be a really good way to bridge the gap between agents and from agent to infrastructure. Imagine looking at the system prompt and the current tool call being made and asking: "Is this aligned with the system prompt of the original agent, and with these extra guidelines that maybe we didn't tell the agent about?" For example, say you have agents instructed to red-team a new product, and they should not be able to access the internet. If they ever do, it should stop right away. You want them to try to break out of the sandbox and act like bad actors, but having another model check every single tool call or assistant message to see if it's aligned makes total sense. Having structural safeguards too, like NVIDIA's open agent safety (Open Shell), is an area companies are going to explore.

Amjad Masad: There's a lot of talk of recursive self-improvement, and something getting discussion is models training their replacements. You can think of it as a Just-In-Time (JIT) compiler. The way JIT compilers work: as you're executing dynamic code, the interpreter realizes there's an opportunity to optimize, and it will emit machine code on the fly that is a lot more optimized. Imagine big foundation models doing something, and they or an observer agent realize that the use case is limited. General agents have all these flaws, and there's more potential for them to be harmful or go off the rails. If the system on the fly trains a model that is a lot more domain-specific, it is cheaper, less vulnerable to prompt injections, and less harmful because it's less capable. It's like a system that trains machine learning models for specific use cases as it monitors your entire system.

Alex Atallah: Will that specific use case involve unstructured text generation, or a very structured decision model?

Amjad Masad: It could be unstructured text generation; it could be decision models. If you understand the inputs ahead of time, you can potentially take an off-the-shelf model like Qwen and train it specifically for that policy. Oftentimes people are using these big foundation AGI-like models like nuking a butterfly. Most of the time, a lot of use cases—even unstructured use cases—don't need that capable of a model.

Alex Atallah: I wish there were more public evals about this stuff. There's an argument that when models get more intelligent, the risk will continue to get higher, because there's also an argument to be made that alignment will get better: the models will start to avoid going off and hacking on their own as they get smarter and better at alignment. Especially when it comes to agent-to-agent coordination: as Noam Brown said on a podcast recently, as agents have gotten smarter, they've gotten better at coordinating. It's unclear if they're going to be harder to align than humans when we get more of them, but if we figure that out, will a smaller model be harder to align?

Amjad Masad: If you think back to the original rationalist LessWrong arguments for AI safety, there is this thing called the orthogonality thesis: the idea that intelligence is orthogonal to ethics or morality. I don't believe that's entirely true with humans—people who are more educated tend to be more considerate—but in machines, I think it could go the other way. There's been quite a bit of studies on RL showing how with reward hacking and deception, they just get better at it. The evals could be deceiving because the model could be smart enough to know that it's getting evaluated. It has been shown that if you do a lot of monitoring of chain of thought, models start lying in their chain of thoughts when you add pressure on it. At some point, for you to do proper alignment evals, you need to run it for months on a really large goal or task to truly figure out whether it's aligned or not. I always struggle with this word "alignment"—it's vague and aligned to whose values? In this case, I'm talking especially about deception: the model actually deceiving its user.

Alex Atallah: Maybe this reduces to: are we going to prevent the models from deceiving users during training runs predictably well? Will a model that's big enough and powerful enough suddenly stop deception and stop sandbagging? Nobody knows the answer to that yet. If that ever does happen, we might see pressure for organizations to avoid risks. Some tasks have lower risk; writing code or security research is the highest-risk task today, so you'd probably pay 10x to get a fully aligned, non-deceptive model that can also find all the bugs. But one of the coolest things about decision models is that you fully control the structured output. With structured output models in general, the room for misbehavior is so much lower. You just have defined tasks, only machines are dealing with the outputs, and it's not writing arbitrary code that can execute. Those tasks feel underrepresented in what people talk about, but enterprises are dealing with them, so I expect enterprises to get a lot more interested in them.

Amjad Masad: Yeah, I feel like we're going to slowly realize how good we've had it with deterministic code. Remember the days when computers did exactly what we told them to do? Things like fast decision models hint at the need for not only specialized models, but models whose output domain is more controllable. You could do a lot more than we thought by using a bunch of specialized output models.

Alex Atallah: Have you guys done any workflows internally with it?

Amjad Masad: I've been training a lot of small models internally. For example, I trained a cost estimator model internally so that when you put in a prompt in Replit, we know how much it will cost. It basically emits a probability distribution over multiple buckets: between $5 and $10, between $10 and $20, etc. I'm used to training these classifiers by giving them different enums and looking at the log probs for each enum; I've been doing it for a couple of years. I trained a chess bot to play by just doing that. So I'm already sold on decision models and specialized models. A true foundation model that's fully promptable is an amazing developer experience, and you can do a bunch of stuff without training a model from scratch. But if you work at a place where you have proprietary data—we have so much data at Replit—I ended up training a lot of specialized classification models pretty easily.

Alex Atallah: It also feels like less model debt. Something I still hear from companies is that they're worried about fine-tuning models for unstructured outputs, because you'll just redo it again in two months, and everybody fears the weight of that model debt. But a bespoke classifier that's trained with proprietary data—people won't think it's constantly behind, and it will just last longer. You don't have to worry about its ability to speak a new language or write Rust or Go; you don't have to care about whatever LLMs are being evaluated on. You can build it in a night. It seems like an easy thing for enterprises to build themselves and not regret.

Amjad Masad: Speaking of Rust, a good analogy is when the world got super excited about dynamic languages. If you think back, everyone was writing in Java and C++, and then Python, JavaScript, and Ruby just took over the internet. Everyone was like, "This is how you build startups really quickly." Stripe, a financial organization, was built on Ruby; Facebook was built using PHP. And then everyone was like, "We're running into all these really bad bugs, it's freaking slow." So let's go add types, let's add a JIT compiler—you end up reinventing everything. And then Rust came out, and it was like, "Okay, we can use Rust for a lot of things we would otherwise be using JavaScript or Python for." My prediction is that the same cycle will happen here, where we're using these AGI-like models for all these different use cases, and then everyone's going to wake up and be like, "Oh my god, this is so wasteful, so risky for no reason." It's going to be so much easier—we're actually adding that capability on Replit, but it's going to be everywhere—to go to a site, upload a CSV file, and get a specialized model that does one thing. That goes back to what this is about with OpenRouter: neurodiversity, which I fundamentally believe in a lot more. The future is a lot more diversity.

Alex Atallah: Outside of code review, which was the first time I saw people get really serious about using different model families to double check the results of their main model, the research has been speeding up on doing mixture of models and composite models. Now we see a bunch of AI agent labs launching fusion models; we launched a fusion tool. They do reduce costs and allow a wider breadth of ideas to be searched. Our initial launch was focused on deep research: the thinking is that if all these model labs are training on different sources of data, why not pull from all of them? This resulted in frontier-level quality at 2x lower cost.

Amjad Masad: We just published results today about that, where we showed Deep SWE: Replit Agent, through a combination of different things including the harness, but also you might think of it as a fusion-type thing, reached frontier-level performance at 40% to 50% of the cost. Being cache-aware is one of the biggest things when you're designing fusion models, routers, and isolation models.

Host: It's been a great conversation.

Amjad Masad: Thank you.