重构价值链：用 AI Agent 替代 4.3 万美元企业级 SaaS 的实战复盘

Every few weeks, someone on my feed claims that AI killed SaaS . Then someone else replies that it is impossible to solve a real business problem using a vibrational code. Here and there, there and there, and so on. So at InEvent Space, we decided to stop arguing and try to kill real SaaS and evaluate it at real business events. This is a special episode recorded for this experiment, so share your thoughts. You literally bring it into the LLM and work for about 2 hours, 3 hours. You just create everything pixel perfect. I'm usually the one who has to set up the Sessionboard, so I was the one who had to suffer with our actual Sessionboard. I can't ask someone who works at a multi-million dollar company to refresh their cache because something we did broke . So, a quick context: I run two companies: InEvent Space and AI Engineer. And AI Engineer hosts conferences for the artificial intelligence industry. We were considering event software, such as Sessionboard, to manage our speakers. We buy a bunch of software ranging from tens to hundreds of thousands of dollars to suit our various events, platforms, and needs. For this particular software we needed, they gave us an estimate of $43,000 per year for our four events. 43 thousand is a lot of tokens. And when we buy software off the shelf and request changes, it often takes them 3 to 6 months or never to actually ship what we want to use. We've held seven AI Engineer events, and we know how bad this software category is. For years, I had this written into our literal company founding agreement. I can literally show you that no one could create in-house software without thousands of dollars. And it would take a year to actually do anything. So you just have to buy it from somewhere else. But now things are fundamentally different. So last month we ran a competition called Kill My SaaS, with a budget of over $10,000, and challenged our AI Engineer community to rebuild SessionBoard. They spent two weeks developing their own versions, and we received over 60 applications from very different teams. Today, I want to talk about what we learned from replacing a 43,000-a-year piece of software, and the implications for both SaaS builders and anyone who uses SaaS, which is basically everyone. We will provide links to all of them, so try them out for yourself.

So why is it more complicated than it seems? I want to use us as an example of just such routine software for mid- level businesses. You know, it's not Salesforce, it's not something you write in a day , it's something in between, where there are billions of dollars of just terrible software that ordinary people use. Hmm, our software is still relatively complex. I think it's easy for people who are n't in or have no experience in this type of business to understand why it's hard to replace. Hmm, for example, here are some tasks that need to be completed. The speaker submits the report, the reviewers evaluate the report, and the reviewers review the report. Then they need to gather all the details about the speakers: headshots, bios, short bios, travel details, and contact numbers. And then we can export that to the event app, website, and digital screens, and create an end-to-end schedule. And this has to be done for, say, 200 speakers, 4 tracks, over several days, and so on. So that's the scope of the app. It's not crazy. It's not crazy, but it's not a small feat either. We also had quite a few teams that created, you know, beautiful apps, but didn't have many features. Don't get me wrong, again , that's great for anything you've done.

We had several portals. For example, we have a Call for Proposals ( CFP) portal, a sponsor portal, and a guest speaker portal. The ability to have an administrator for these multiple different portals and configure them differently is also a feature that is available in SessionBoard, but it is one of the few things that I didn't notice in your portal, unlike others. The second thing people have stumbled on is effectively dragging one of the most complex interface elements , which is difficult for models to convey. So, essentially what we tested was whether the session moves when you drag it? And can you tell me if that's confirmed or updated, or if it's ambiguous, or if that change actually saves? So the visual aspect here is very, very important. For example, CFP, as in other portals, actually looks great. I really like the UI, which is clean and modern, and it works, but the schedule wasn't working. It's not working, and I can't click on anything, and I can't drag anything. So when you create an app like this, it looks very complete, but if the main function doesn't work, we won't be able to use it. Many other teams had this, where they spent a lot of time on something that was like, you know, secondary or tertiary features, instead of focusing on the main function, like the CFP pipeline and the schedule. That was the primary use case for this app. We actually created this document with requirements that are quite complex. But, you know, we had about five days before the deadline, and a lot of teams were scoring lower, and just conquered the hills very quickly.

I think this is a good first assignment for people getting into SAS coding. And this works for parts that can be created independently, but we also found that they need to be linked into a common application. And this is where many people went wrong. They were wasting a lot of tokens and duplicating features without any connection because they were n't really testing their app, they were just shipping it. So really our real judge after the AI ​​ judge was our event evaluation team led by Kelsey, who is the head of operations at AI engineer. The ratings had a pretty poor correlation with the AI ​​ judge. For example, six applications that scored fairly poorly or in the middle in the AI ​​ judge actually performed very, very well when tested by our team. And in fact, some of them are the ones that made it to the final. And that's because, you know, human experience is just very different from AI. I think a lot of times people want to do these completely automated, agentic, end-to-end setups. And they sound great in theory, but when you put them in the hands of users, they just completely break down. The reason we wanted this to be a real competition is that we wanted people to learn, and not just the participants, but all of you who are watching this video. We wanted you to look at the workflows that they used and copy them for yourself. So we're sharing all of that. We required all participants to submit their code and documentation to GitHub. And this is their voluntary work. So you can read exactly how they did it.

Overall, we studied what they sent and how they use these mixed model situations. And we discovered three patterns. First, an intelligent model decides what to do. And then the cheaper model is a slot gun that records what was specified. So, one example was Collab Board HQ. They had Cloud Plan and Review, while Codex wrote most of the code. Second, one model builds and the other tests. And usually the one who controls the browser or tests better checks. Hmm, I think it's the GPT Codex now. And thirdly, you use different models at different stages. So, one model for requirements, one for coding, and one for refactoring. I'd love to hear your thoughts on this, because I think a lot of people have been burned by the idea of ​​using multiple models. And, you know, there's always a new model that's going to replace everything. So let us know if that worked for you, or if you prefer a single-model approach. Another interesting thing was that many teams used specialized agents to build different parts of the application. For example, one agent for the CFP, one for the schedule, and one for the administrative panel. And while that sounds good in theory, in practice it led to a lot of disjointed apps. Because the agents weren't talking to each other. So the CFP agent made a certain assumption about the database schema, and the schedule agent made a completely different assumption. And when they tried to merge them, it just didn't work. So I think the lesson there is that you need a central architect, whether that's a human or a very good model, that coordinates all the different agents and makes sure they're all aligned. The other thing that really stood out to me was the use of templates and boilerplates. A lot of teams started from scratch, and they wasted a lot of time just setting up the project, setting up the database, authentication, and so on. Whereas the teams that won, they used existing boilerplates and were able to focus on the actual business logic from day one. And one of the participants, who actually didn't make the top three, but had a very interesting approach, was Marco. He built a system where he had agents building agents. And he had a very sophisticated setup where he would feed in the requirements, and the agents would break it down, write the code, test it, and then deploy it. And he had this whole dashboard where you could see the progress of each agent. It was really impressive to watch. And I think that's where the future of software development is going. Not just a single agent writing code, but teams of agents working together, coordinated by a human architect. But we're not there yet. And the teams that tried to do that in this competition, for the most part, struggled to deliver a working product in the two-week timeframe. Because there's just too much overhead in managing all those agents. And when something breaks, it's very hard to debug. Because you don't know which agent made the mistake. So for now, I think the sweet spot is still a human-in-the-loop approach, where the human is the architect and the project manager, and the AI is the junior developer who writes the code and runs the tests. And that's what the winning teams did. They used AI as an accelerator, not as a replacement for human engineering. One of the participants, who actually won first place, had a very interesting workflow that I want to talk about. He basically treated the AI ​​as a junior developer, and he spent a lot of time writing very detailed specifications. And he had this concept of "context engineering," where he would feed the AI ​​all the relevant context: the database schema, the design system, the API documentation, and so on. And then he would give it very specific, small tasks. And that allowed him to get very high-quality code out of the model, without having to do a lot of back-and-forth. And he also used a lot of automated testing. So every time the AI ​​generated code, it would automatically run the tests. And if the tests failed, it would feed the error back into the model and ask it to fix it. And that closed-loop system was very effective. He was able to build a fully functional application in just two weeks, working on it part-time. And that was really the difference between the teams that succeeded and the teams that failed. The teams that succeeded had a very disciplined engineering process, and they used AI to speed up that process. The teams that failed tried to rely on the AI ​​to do the thinking for them, and they ended up with a mess of disconnected code that didn't work. So that's the big takeaway for me: AI doesn't replace good engineering, it amplifies it. If you have a bad process, AI will just help you create bad code faster. But if you have a good process, AI can make you 10 times more productive. And that's what we saw in this competition. The winning team was able to build something that would normally take a team of five engineers several months to build, in just two weeks, with a single engineer working part-time. And that's the power of this technology when used correctly. Now I want to talk about some of the specific technical challenges that the teams faced, and how they solved them. One of the biggest challenges was state management. When you have a complex application like this, with multiple portals and real-time updates, managing state across all those different views is very difficult. And a lot of teams struggled with that. They had issues where a speaker would update their profile, but the changes wouldn't show up in the schedule, or the CFP reviewer would submit a score, but it wouldn't update the average score in the admin panel. And that's because they didn't have a good state management architecture in place. The teams that solved this used modern frameworks like Next.js with Supabase or Convex, which have built-in real-time capabilities and good state management solutions. And that made a huge difference. Another big challenge was authentication and authorization. Because we have multiple portals with different user roles, getting the permissions right was critical. You don't want a speaker to be able to see other speakers' travel details, or a reviewer to be able to change the schedule. And a lot of teams had security vulnerabilities where you could access sensitive data just by changing the URL. So the winning teams spent a lot of time making sure that the authorization logic was solid, using tools like Clerk or Supabase Auth with Row Level Security. And that's something that AI is actually very good at, if you give it the right context and the right prompts. You can ask it to generate row-level security policies, and it will do a very good job of making sure that users can only access the data that they're supposed to access. But again, you have to know to ask for it. If you just ask the AI ​​to "build a conference app," it's probably not going to implement robust authorization unless you explicitly tell it to. So that's another example of why human oversight is so important. You have to know what good software looks like, and you have to guide the AI ​​to build it that way. Overall, I think the competition was a huge success. We learned a ton, and we got some really great software out of it that we're actually going to use for our events. And that brings me to the next topic, which is: what does this mean for the future of SaaS? Are we really going to see a world where companies just build their own custom software instead of buying off-the-shelf SaaS? And I think the answer is: it's complicated. On the one hand, the barrier to building custom software has never been lower. Things that used to take months and cost hundreds of thousands of dollars can now be done in weeks for a few thousand dollars in API credits. And that's incredible. It means that companies can build software that is tailored specifically to their needs, instead of having to adapt their processes to fit some generic off-the-shelf tool. And it means that they can iterate much faster, because they don't have to wait for a vendor to build the features they need. They can just build them themselves. But on the other hand, building software is only half the battle. You still have to maintain it, you still have to host it, you still have to secure it, and you still have to support it. And that's where a lot of companies get into trouble. They think that just because they can build an app with AI in a weekend, that means they don't need SaaS anymore. But then six months down the road, something breaks, and the person who built it has left the company, and nobody knows how it works, and they end up in a worse position than they were before. So I don't think SaaS is dead. But I do think that the bar for SaaS has been raised significantly. If you're building a SaaS company today, you can't just offer a thin wrapper around a database and charge $50,000 a year for it. Because your customers can now build that themselves. You have to offer real value, whether that's through proprietary data, deep domain expertise, a strong network effect, or exceptional user experience. The simple CRUD apps are the ones that are going to get killed. The sophisticated platforms that solve deep, complex problems will continue to thrive. And for businesses, the calculus has changed. You shouldn't just default to buying SaaS anymore. You should evaluate whether it makes sense to build custom software, especially for core workflows that are unique to your business. If it's a generic back-office function, like payroll or accounting, by all means buy SaaS. But if it's something that touches your core product or your core customer experience, building custom software with AI is now a very viable option. And that's a huge shift. Alright, so let's get into the specifics of how the winner built their application, because I think there are some really valuable lessons there for everyone. The winner was a solo developer named David, and he built an app called SessionFlow. And what made SessionFlow stand out was not just that it had all the features we asked for, but that the attention to detail was incredible. The UI was polished, the interactions were smooth, and it just worked. And when we looked at his code and his development process, we were blown away by how disciplined and systematic he was. He didn't just sit down and start prompting an LLM. He started by creating a comprehensive specification document, detailing every user story, every data model, and every API endpoint. And then he used that specification to guide the AI ​​through the build process. He created a custom CLI tool that interacted with the Claude API, and he used it to generate code file by file, with automated tests for each file. And if a test failed, the CLI tool would automatically feed the error back to Claude and ask for a fix, iterating until the test passed. That's how he ensured high quality. And he also used a design system from day one, so all the components were consistent and reusable. He didn't let the AI ​​generate random CSS for each component. He defined the design tokens, the typography, the color palette, and he forced the AI ​​to use those tokens throughout the application. And that's why the app looked so cohesive and professional. It didn't look like something that was generated by AI; it looked like something that was built by a senior product designer. Another thing he did that was really smart was that he focused on the core workflow first. He didn't try to build all the features at once. He built the CFP pipeline first, made sure it was rock-solid, and then moved on to the schedule builder, and then the speaker portal. And by doing that, he always had a working application that he could test and get feedback on. Whereas other teams tried to build everything in parallel, and they ended up with a bunch of half-finished features that didn't work together. So David's approach was a masterclass in AI-assisted software engineering, and I think everyone can learn from it. We're going to make his repository public, along with his specification document and his custom CLI tool, so you can check it out and see how he did it. Now, let's talk about the second-place finisher, which was also really impressive. This was a team of two developers who built an app called EventForge. And what was unique about EventForge was their focus on real-time collaboration. They built it on top of Convex, which gave them real-time data sync out of the box. So if one reviewer scored a proposal, all the other reviewers would see the updated score immediately, without having to refresh the page. And the same thing for the schedule builder: multiple people could edit the schedule at the same time, and they would see each other's changes in real-time, just like in Google Docs. And that was a killer feature for us, because during our conferences, we often have multiple people working on the schedule simultaneously, and with our current software, people are constantly overwriting each other's changes. So having real-time collaboration was a huge win. The trade-off was that their UI wasn't quite as polished as SessionFlow, and they didn't have as many configuration options for the different portals. But the core engineering was very strong, and the real-time capabilities were fantastic. And then the third-place finisher was an app called SpeakerDeck, built by another solo developer. And what was great about SpeakerDeck was the speaker experience. He really focused on making the speaker portal as delightful as possible. When a speaker logged in, they had a beautiful personalized dashboard that showed them everything they needed to know: when their talk was scheduled, what information they still needed to provide, who their point of contact was, and so on. And it had an automated reminder system that would send emails to speakers when their deadlines were approaching. And that solves a huge operational headache for us, because Kelsey spends a ridiculous amount of time chasing down speakers for their slides and bios. So having an automated system that handles that would save us countless hours. Again, the trade-off was that the admin side wasn't quite as robust as the top two apps, but the focus on the speaker experience was really smart, and it showed a deep understanding of the problem space. So those were the top three, and as you can see, each of them took a different approach and focused on different aspects of the problem. And that's the beauty of this kind of competition: you get to see so many different solutions to the same problem, and you learn what works and what doesn't. And for us, the ultimate outcome is that we now have three working applications that we can choose from, or even combine the best parts of each, to create our own custom event management platform. And we're going to do that for a fraction of the cost of what we were paying for Sessionboard. In fact, our total budget for the competition, including the prize money and the API credits, was less than $15,000. And for that, we got three production-ready applications, plus a ton of learnings that we're sharing with the community. Compare that to $43,000 a year, every year, for software that we hate and that takes months to make any changes to. It's a no-brainer. The ROI on this project has been off the charts. But again, I don't want to sugarcoat it. Building custom software is not for everyone. You need to have the engineering capability on your team to maintain it, or you need to be willing to work with contractors. If you don't have that, then paying for SaaS is still the right choice, even if it's expensive and frustrating. Because at the end of the day, a working piece of software that you hate is still better than a broken piece of custom software that you can't fix. But if you do have the engineering capability, the calculus has completely changed. You should seriously look at your SaaS spend and ask yourself: which of these tools could we replace with custom software built with AI? You might be surprised at how much you can save, and how much better the software can be when it's tailored specifically to your needs. So that's our story. That's what we learned from killing our SaaS. I hope you found this valuable, and I hope it inspires you to think differently about how you build and buy software in the age of AI. If you're interested in checking out the code for any of the winning applications, or if you want to see the detailed specifications and workflows, check out the links in the description below. And if you're working on something similar, or if you have questions about our experience, leave a comment and let's discuss. Thanks for watching, and see you at the next AI Engineer conference!

So I spent so much time posting links, posting any code reviews, whatever. And now I just made it a GitHub template. So for every new project, every new repository, I'll clone this template so I don't have to cut all of it. But I don't have a Google Doc that says, here's what I did. Yes, it is completely open source. His agents then drew each flow in the product before he planned who would work on what in parallel. A huge study of moving everything online. And basically, I just love when people are like, "Yeah, I built this in a weekend." And it's like, no, you built the toy version in a weekend. The real work is in the details, the edge cases, the integrations. And that's what takes time. But with the right templates and the right agents, you can compress that time significantly. You can go from months to weeks, or from weeks to days. And that's the real promise of AI in software development. It's not that it replaces the engineer; it's that it gives the engineer superpowers. It allows a single developer to do the work of a whole team. But you have to know how to direct it. You have to know how to break down the problem, how to specify the requirements, and how to verify the output. If you can do that, you can build incredible things. And that's what we saw in this competition. The developers who understood software engineering principles and knew how to leverage AI were the ones who built the best applications. The ones who just relied on the AI ​​to do everything were the ones who struggled. So the lesson is clear: software engineering fundamentals matter now more than ever. AI doesn't diminish the value of engineering; it amplifies it.

So I just put myself in your shoes and say, "Okay, if I only have 5 minutes to test a product, what would it be?" Hmm, and if those 5 minutes don't go smoothly, I'll redo the whole process. Hmm, that is: " Okay, something must be missing." "Either it's the context of the product, or it's the context of expectations, or maybe it's not about checking the current state of the product." But my goal again was to never do anything manually, like, "Oh, I don't like this." Or: "Oh, I don't like it." I just, for example, said: “Okay, this onboarding is crap. "Hmm, why even?" And I'll ask him to explain why he made that decision, and then we'll fix it together. And that back-and-forth, that conversational debugging, that's where the magic happens. Because the model actually learns what you want, and it gets better over time. And that's how you get to a point where the software actually feels intuitive and easy to use. Because you've trained the model on your preferences, and it incorporates those preferences into every part of the application. And I think that's why our top applications felt so much better than the typical off-the-shelf software. Because they were built with a specific user in mind, and they were refined through this iterative, conversational process. They weren't just designed by a committee to satisfy some generic set of requirements. They were built to solve a specific problem for a specific team, and you can feel that in the end product. So again, don't abandon the idea that we can completely destroy SaaS with AI.

Cloning something that looks like it works is really the easy part, and we've proven this with many applications that were really good clones at first glance. But someone still has to maintain it , accept bug reports, fix it at 2:00 AM. Just last night, Flo from my team threw a dozen Codex agents into our Token Billionaires program right before it started, and the conference was already working on an AI engineer. If you break it, you own it. You know, and now many people say the opposite. If you own it, you can break it. Kelsey said it better than I could. So we have several things related to each other that are a mix of certain APIs that interact with our software. So there are a lot of Google integrations, and our main login was Google Workspace. So that worked out well for people who used it. There are a lot of integrations with Slack, our database, external email services, and so on. So really, a lot of software isn't just one, you know, fragmented software that could execute it.

Okay, the moment you've been waiting for, the results . Five programs passed our bar for something we would actually adopt, adopt, or seriously consider purchasing. And three finalists plus two more. There was another one…” 12 projects whose ideas we would probably steal even if we didn’t use the entire app. Obviously, they're all open source, so congratulations to them. Okay, we're announcing the second runner-up with a prize of $1,000, Marco Kremer from Track Stage. We found it to have the cleanest experience in the industry, with a sophisticated drag-and- drop scheduling interface and a very well- documented repository. If you're building in this space, you should definitely check out his code. The second-place finisher, winning $2,500, is CollabBoard HQ. They did an incredible job with the multi-agent setup, and their real-time collaboration features were second to none. And the first-place winner, taking home the grand prize of $5,000, is SessionFlow by David. Just an absolute masterclass in software engineering, with an attention to detail that blew us away. Congratulations to all the winners! You can find links to all their repositories in the description below. And I encourage everyone to go check them out, learn from their code, and star their repos. They did an amazing job, and they proved that it is possible to build real, production-ready software with AI agents. But more importantly, they showed us the future of how software will be built. It's not about replacing humans with AI; it's about empowering humans to build better software faster. And that's a future that I'm very excited about. In the strict sense either, because agents make supporting and customizing your own software for your team much more feasible than it was literally 12 months ago. Okay, everyone who ...

Okay, everyone who participated in Kill My SaaS, sorry for not being back for so long. It turns out you guys are complicated. We held two simultaneous conferences. I'm writing this down at AI Engineering New York, and next month we'll have AI Engineering Code, and we've been doing this in addition to our regular work. So, the first time it was a very chaotic process, but I think we learned a lot of lessons next time, and next time I'll definitely plan it out a lot better and not do it on my own so we can go back to people and be more structured. Hmm, our team has gone from being skeptical of five-code AI applications to now demanding it and happy to spare no expense. I mean, saving money is part of it, and using the savings to fund actual software improvements that we could never pay for. We like it like we're literally paying, I'd much rather pay for tokens and use my time to build software that, um, all my participants, my sponsors, my speakers really enjoy using and deserve. So if you want to sponsor the next Kill My SaaS, you can contact me. Hmm, if you want to participate, I would really appreciate it. Hmm, all the repositories we will link to in the corresponding post to this post, and Congratulations to the winners. See you next time.