I think the story kind of starts at COVID, you know, I basically did a Corona CLI thing that went viral. So COVID was at its peak. I was traveling a lot being in DevRel and whatnot and I think Greg Brockman and Sam Altman ended up giving me access to GPT-3 early. So since then I've been an AI engineer, right? And the first thing I did, in fact, I was just looking at it. It was July 2020. And Greg sent me a message like what is the use case? What What are you going to use this API for? And I told him I'm going to suggest the next line of code like a code snippet, right? This is year and three more than a year before GitHub Copilot was a thing, right?

So I started building this thing called CLAI. I've always been a big fan of building CLI, so you know, silly side project Cly, right? And uh I think that has eventually became command code now. Like, you know, our short history is we ended up building an AI cloud. Now, everything is an AI cloud. It was called LangBase. Uh it grew quite big. Uh 1.2 billion agent runs a month. Ended up building an a memory infrastructure or whatnot. And now we've sort of like pivoted into this feeling that there is only one type of agent, and that is a coding agent. It can do it all, right? So, why hide that capability behind some memory system or some primitive or whatnot? In that uh 6-year-old old code base has eventually turned into this thing we call command code, right?

If you spend a billion tokens on Claude, on GPT, on DeepSeek, you discover a lot of discrepancies across the stack in different full stack systems like DeepSeek is not that good at Go code for some reason. And all of those, I started calling them variations. And we discovered a deterministic pattern across those variations. So, when you're running a coding agent, tool call on a very basic thing is like, you know, if you're trying to run a bash or shell tool to discover something like you are trying to list what the directories are, you're trying to read through a bunch of different files. User has asked about a question about, you know, maybe your how is authentication built in this repository. And your coding agent is right now trying to figure out with a bunch of tools that are pre-cooked in that I need to list all the files, read a bunch of them, explore them, and then answer the user.

So, what of what what a lot of open models are doing is I think they somehow suck at tool calling, right? And the pattern is not like super broad, right? It is like, you know, when uh is very specific to DeepSeek. So, DeepSeek V4 Pro has this weird alpha male energy where whatever it sends you, it thinks that that is the right thing to do. And if you uh if it is sending you wrong schema of the tool calls, and you send back a Zod error, it doesn't listen to you. It would repeat that same thing for like 56 times on average in a billion tokens. Where you'll be like, "Why are you Why is it doing that?" Right?

I have no idea. Like, you know, as a programmer, I would think that, you know, if my tool call is failing, all I have to do is send some schema back, like Zod error, and an LLM that is smart enough, right? And it will just do it, right? My gut feeling is it might be a hot take the because they are I think a lot of these open models are built or trained in a system where they are learning from data, which they consider really high quality. Like, they're learning from a better model than them or something like that, right? And their training is whatever you are being told is right, it's correct. So, their entire nature is whatever I am telling you is also correct, so don't try to correct me.

And the thing that we discovered was like it was sending in whe- you have some tool like a shell tool and it has some parameters or arguments that should be optional, it would send some weird thing on that like an empty object or or a null where it is it doesn't belong and Zod being super strict would just trip up and send back that error. And it will get back the same result again and again. Same tool call, right? So instead of sending back that error, I ended up repairing that, right? It started with just like 3, 3,200 lines of four repairs. Think of it this repair logic like, you know, database migrations. You know, you have one migration per file. So I ended up creating repair files. Like if you see something like this where, you know, it is emitting, you know, JSON strings type of data when I actually wanted an array, I can determinately deterministically fix that to an array, right? And when I do that, I will not only just send back the result, I will also send back a note, a hint, a repair hint that, you know, you should have sent me this type of data, but here is the result anyway. Think of it like you're teaching somebody how to drive a car. And they're about to, you know, hit another car. Instead of telling them what to do correctly, you'll first try to save them and then you will explain like why you saved them, right? Like what they should have done in the first place. I think a lot of these models really like this repair logic because what we saw is the moment you send the result with the repair logic, right after that the third tool call is fixed. Instead of, you know, and it it all of a sudden becomes super smart. It understands like, okay, I got the result what I was looking for and I'm I'm going to do this. And it shows up in so many different places as well. Like for example, it is trying to read a file and it is not giving you the offset of are you trying to read the 100 lines from the top or at the bottom of this file. So, I just make a judgment call like, okay, it's the first time it's going to read that file, so let's give it the first 100 files of first 100 lines of that file, right? And then it realizes very quickly, oh, I I was actually trying to read a log file and then needed the last 100 and it can very easily understand instead of, you know, those 50 plus on average tool call failures.

And something like this should be very obvious to developers, but what's happening is a lot of developers actually use cloud code. They hack their, you know, change the base API endpoint and the, you know, API key and they try to use cloud code as the hardness for open models. And cloud code, uh, you know, you know, they hide a lot of the errors behind control O, right? So, you don't even know that, you know, you have like 50 plus tool call failures plus per session. You you're just sitting there and you're like, oh, why is DeepSeek so slow? And they are they it's in their vested interest to not fix that. Like, why they don't care about open models. They didn't build their coding agent for open models. So, it works out really well for them, but for the common thing that the common team across Twitter is, oh, it's so good and oh, it's so bad, it's super slow. So, it I feel like this always ends up being a tool call hardness issue than, you know, an actual model issue.

Yeah, I think the first thing again me being me, I thought this is just DeepSeek. Then I looked at our, you know, logs for last 30 days and Kimi is doing exactly the same thing. Then we fixed the Kimi models and we fixed MiniMax models and now we have like 16,000 different repair you know, variations across hundreds of billions of tokens. We are doing anywhere from 600 billion tokens right now. And the data on failure of those tool calls super super important. In overall, this goes from a model that was practically completely not useful like DeepSeek V4 flash to something that can actually compete with this, right? Like and this is like more of a vibe check when I pushed out this update. One of our investors, Tom Preston-Werner's fund, PW their GP, he was like, "What did you do? Why is DeepSeek V flash V4 flash super solid now?" Like the vibe of the model completely changes. It starts doing things in a different way. I don't know if you've seen this, but like if you run any coding agent with permissions on the models are actually dumber. And if you run them without, you know, the complete bypass of permissions, they do much better. Even if you like sit through those yes yes yes accept or whatnot, you will see that, you know, the model ends up getting steered in the wrong direction because of the slowness of permission blocks or whatnot. Maybe that's not how the models are trained or whatnot, but it's it's the same thing with models. Like if they are seeing a lot less tool call errors, they are much more creative. They are they can explore a lot and they can continue a lot longer. Like one of the our users has actually done like I think 70 billion tokens on Deep Seek. I was looking at his data. He broke our usage page. That's how we discovered him. And he's like, "I constantly run Deep Seek with Command Code for 12-plus hour-long sessions." I personally have not done that, right? So, when I think a lot less tool calls are happening tool errors are tool confusion is happening. This repair logic kind of blows you away like, you know, how good open models can be overall.

But, one of the most interesting things that just recently happened was we have been able to apply the same thing to design slop. You know, like you know, that indigo purple gradient thing that all LLMs do?

There you go, right? I think everybody just knows So, I love the purple color. And I would like to point out that it's the indigo slop that is happening, not the purple slop. Right? But, but what we have found is we can deploy the same model to fixing design slop across the hundreds of billions of tokens that we have done. We looked at the numbers and we found that obviously we chatted with a bunch of amazing designers with amazing design taste. And we found out that it is also very similar a problem. There's a finite set of things that most LLMs do. And if you can give them like a compositional framework of sorts, like you can repair their design thinking, the same thing applies, right? Like for example, these are like probably 10 big rules you can put them in a skill file or anything. And you will see your design get better. We've seen this repeated across different LLMs, commercial or open, whatsoever. It's the same repair tool logic of sorts, right? Like you You basically guiding your coding agent that, you know, you're not going to do this, this, this, this, and this. And if it does, you are deterministically fixing it. Like one of the things that you might find really good here is first one was this the work pattern first composition where when you ask a model go and design me this dashboard they generally do not think about the intention behind that design and they just uh slop you with that, you know three cards in a row and, you know, one border on the left side or top side or what not. That's a pretty common thing. Uh if you give them a very simple framework of, you know, what type of surface area are we looking for, which is literally just these seven patterns, they do really, really well. And things like

Pe- People I think you could you could think of it like, you know, when a designer is designing a dashboard they're thinking about this is a monitor surface, right? Like we're trying to monitor That is the intention behind this, right? And this is across the chats we had with a bunch of amazing designers, this is how they work. And we thought if we could turn this into a slash design skill see how the limbs will repair themselves. We also discovered something really, really we're like I personally don't use OKLCH, but apparently LLMs are really good at it. And if you see them using HSL or something they are they they they they don't actually are able to control the lightness in HSL very quickly, but on to human eye it's very, very easy to see like this color and this color do not look the same. But if you force an LLM to use OKLCH, they can control the colors palette really really well compared to any of other things. And this is what a lot of my designer friends do as well.

So, yeah, but it it's I can't shake this feeling, but it's like I actually wrote about this just last night. This is from last night. Like I think they're they're the design slop thing somewhere here. Like this is new, so I'm struggling to find where that is. They We only have like 24 reference documents, 10 you know, design smells, and seven patterns that we saw across different designers. And when we designed a bunch of different landing pages and got those designers to look at them, it takes about 1.5 seconds for them to spot, "Oh, this is from AI and this is somebody human looked at it." Right? So, the differences that they spotted, when we wrote them down, they are just deterministically fixable patterns. Like you can convert HSL into OKLCH, or you can force an LLM to do that, right? And it's like a feels like you can fix 90% of design slop, which is not a capability gap. It's more like a contract gap in what your hardness is telling an LLM to do versus what your user is saying. User is always going to say, "Fix my design, make it prettier, make it pop," or something like this. If you can give them a framework of this is what the design taste of a really good designer is like, they will pick this type of color scheme, they will pick they will think about intent before starting to implement that landing page. What is the intent here, right? It's a short contract, but it's makes your design slope really, really minimal.

Yeah, it's it's like for example, this is the landing page of our documentation, right? As a developer, I would not have gone through the trouble of creating all of this, right? But when I fed that to our design skill, so command code now comes up with a comes bundled with a slash design skill, which has like these all of these references in there cooked pre-cooked in. Like I was just last night fixing our deals that we have like going 2.6 is you know, 50% off. So, I fed that a very basic screenshot of, you know, all of this mess, and this is what it converted into. In it to me it looks really really good. I'm like, okay, this is this feels like, you know, like a ticket that you could print up you know, on a I don't know, like a cinema ticket or what not in from deal that it basically understood the intention behind this thing and tried to to recreate that design. When I as a programmer only told it, here's a deal, here's thedata, just add that to our docs, right? So, giving that framework of you know, this is not like tool calling or anything. It's like a really nice framework of thinking that LLMs have this capability cooked in. They can organize their thoughts really really well. They can design really really well if you just give them the right way to think about those things, right?

And command code actually started with this feeling that I was using command code a lot more than other coding agents personally. And then a couple of team members started adopting it. And we started ending up building this meta-neuro-symbolic model called, you know, Taste One. The thing it does is like I have a lot of experience with code. Like, I've been I think writing code for 27 years or something, right? After publishing 300 plus open source projects, you get to have a lot of opinions on things. And I mostly find myself working on things that are super cutting edge, so there are no docs that an AI agent can go read or whatnot, right? So, at that time I feel like, you know, my opinions matter more than what an LLM can actually find or what you can do with rag or whatever. So, I ended up encoding this behavior uh in meta-neuro-symbolics uh neuro-symbolic architecture where if you learn something from me, document it for me like a skill, right? And we started calling it Taste, right? If you see me prefer pnpm a lot, but publish or link my local CLI with npm global link, right? They'd learn that I prefer pnpm for installing packages in almost every other thing, but when I'm linking my CLI locally, I'm using npm. So, these type of things and learnings and [snorts] that eventually ended up becoming taste files, which are very similar to skill files. You can think of it like command code automatically learns from you on a per repository basis. So, your team, right? And it builds a library of skills, which is quite less verbose, right? They're not like, you know, everything is not in there. It's like things that it sees as like repeated preferences and patterns across your work, right? And it could be coming from so many different coding agents or whatnot. When we When you merge something into main, that is when we can trigger what were your accepts, edits, and rejects overall.

Yeah, I there are like two directions right now that we sort of own. Uh this thing is again this is like a CLI, right? It it is like full-fledged coding agent does everything that you can expect any coding agent to do. It has both uh you know, commercial models or open models. It's just that that we have found our PMF of source in the open models market more than you know, Claude is actually really really lenient with two calls. So even if you know, your coding agent harness messes up, it can figure out that oh, I'm I'm being sent this error and can fix itself. Not the case with uh you know, uh open models, but the taste thing is uh still there. Like it basically sort of like works like this. This is a very common uh site that I have. When I'm using something, please use TypeScript. It's using TSE. I I want you to use TS uh TS up, right? It's using some different framework of testing. I prefer Vitest. And it's like a lot of you know, back and forth in getting your coding agent to do what you wanted done in the first place. But with taste as taste is like a meta neuro symbolic model. Like I have a lot of uh I tried to like based on the feedback I've hidden it very well in docs somewhere. So you know, developers don't have to go and read the really silly things there.

Right? The entire KL divergence loop that we look at. Like if an LLM already knows about something, it should not end up in your you know, skill or taste file. That is absolutely useless context, right? So it basically does all of this weird thing where once you actually go through Where is the link? Once you actually just go through all of this, once you have built a CLI or an API or a front-end project with command code, it actually ends up learning a bunch of those rules that are automatically managed for you. So for example, if you are using Commander for building CLIs and now in this particular project you start using Meow, it will replace that for you. The entire idea is that your skills are being automatically learned and automatically managed and they're absolutely transparent. They are in your repository, not in our model. So, you're reviewing it in every PR. You're looking at like, "Yeah, I don't want to you know, save something like this or what not." And they're never stale. A lot of issues that I feel a lot of people face are because their agent.md or cloud.md has some wrong information when they sat down. Uh this is like one thing I discussed when we launched we announced our, you know, $5 million seed, that a lot of people when they sit down, they think of the rules or what not. They think in the terms of grandiose things, right? Rules are like, you know, like actually zoom in. So, this is like the difference between our skill or rules file or our TS file, right? This is what you are writing down. And TS is what is continuously being learned from your prompts or your edits and they're being stored in the same markdown file for you, right? And updates are when you remember to do it. In most of the times I've seen we we as humans we sit down and we do this grandiose thing. Like, you know, use this. I always prefer that or what not. And TS are a lot of micro decisions, not too broad. When you are doing this, when you are running up/pr Q command, sometimes Emmet prefers that, you know, you basically fetch the latest from main, rebase it, send a PR on a branch and then go back to the main branch. That is what my PR workflow is. That is not what my PR file says. So, it automatically goes and fixes it, right? That I it has seen me go back to the main branch again and again. It's like, "Okay, it's just one liner for it, but improves my workflow without me having to take care of it." And over, you know, over the time it basically compounds a lot.

The At the very basic layer, taste is the highest order bit, which is managing your skills and rules. Skill Taste is this automatic engine of sorts that is that is creating skills for you, making sure they're not stale, and you can obviously go edit them yourself as well. It And overall, it actually looks like this, like after looking at I think I probably have built more than 70 CLIs with command code so far. And this is the entirety of my taste for building CLIs. This little thing, right? So, it knows that for example, I use PNPM only build dependencies thing. It knows that I always prefer starting from 0.0.1. It's like those silly little things. And this is like as all I have to do is I have to do this NPX taste pull. We have the taste package as well, by the way, right? Nice. Yep. It's And it just pulls this particular file, puts it in your repository, and then all you have to do is ask any coding agent and follow my taste of building CLIs. And build me a CLI that does this, this, this, and that. And then by the end, show the taste compliance. And it would go Any LLM would just go through all this list and be able to figure out I I was supposed to use this.

I can probably share just a little bit about what uh our road map looks like, where we are headed next, right? So, we are going to open source command code very very soon. I'm hoping we can announce that on the AI engineering conference in SF.

The idea is you should be able to modify any part of command code irrespective of where our business model is headed, right? And the other idea that I'm super subscribed to right now is that we will not turn this into a soup of, you know, 1,500 you know, models that you decide what you do with them, right? I think of it like, you know, so there are there are like three different philosophies, right? Uh one is it's like Windows and every game works with it. You know, I think Open Code is like that. Every model works with Open Code. One is like Linux where you build your own drivers like Pi. You can build anything with Pi, right? Uh with Command Code what I'm thinking and what the team kind of is going for or towards is, you know, build it like Apple. It will have the best of the best models, both open and closed. It will not have every model, but it will be hackable in any way, so you would be able to put in your local model if you wanted to or whatever, right? So, that's where we are headed like we're about to open source it very, very quickly. Uh very, very soon. So, yeah, pretty excited about that.