So I want to talk about first the kind of weird constraints that LLMs have and those weird constraints are sort of what we have to base a lot of our work around. Now, there's a guy called Dex Hy who runs a company called Human Layer, and he came up with this idea, which is that when you're working with LLMs, they have a smart zone and a dumb zone. When you're first kind of like working with an LM and it's like you just started a new conversation, you start from nothing. That's when the LLM is going to do its best work because in that situation, the attention relationships are the least strained. Every time you add a token to an LLM, it's kind of like you're adding a team to a football league. You think of the number of matches that get added every time you add a team to a football league. It just go scales quadratically. And that's because you have attention relationships going from essentially each token to the other that are positional and the sort of meaning of the individual token. And so this means that by around sort of 40% or around I would say around 100k is kind of my new marker for this because it doesn't matter whether you're using 1 million uh context window or 200k. It's always going to be about this. It starts to just get dumber. So as you continually keep adding stuff to the same context window, it just gets dumber and dumber until it's making kind of stupid decisions. Raise your hand if that feels familiar to you. Yeah. Cool.

So this means that we kind of want to size our tasks in a way that sticks within the smart zone, right? We don't want the AI to bite off more than it can chew. And this goes back to old advice like Martin Fowler in refactoring uh like uh the pragmatic programmer talks about this. Don't bite off more than you can chew. Keep your tasks small so that you as a developer, a human developer don't freak out and don't start acting and going into the dumb zone.

Another weird constraint of LLM is LLM are kind of like the guy from Momento, right? They just continually forget. They could just keep resetting back to the base state. Let me pull up this diagram. I sort of I I I really should use slides, but I just prefer just like randomly scrolling around a infinite uh TL draw canvas. Thank you, Steve. Um, so let's say another concept I want you to have is that every session with an LLM kind of goes through the same stages. You have first of all the system prompt here. This gray box here is essentially the stuff that's always in your context. You want this to be as small as possible because if you have a ton of stuff in here, if you have 250k tokens, like I have seen people put in there, then that you're just going to go straight into the dumb zone without even being able to do anything. So you want this to be tiny. You then go into a kind of exploratory phase. This blue is sort of where the coding agent is going out and exploring the codebase. Then you go into implementation and then you go into testing and kind of making sure that it works, running your feedback loops and things like this. Raise your hand if that feels familiar based on what you've done. Yep. Sort of the like the the main cornerstones of any session. And when you clear the context, you go right back to the system prompt. Bof, you go right back there. So you delete everything that's come before.

And raise your hand if you've heard of compacting as well. Yeah. Okay. There are some people who've not heard of compacting. So let's just quickly show what that means. For instance, I've just been having a little chat with my LLM. ... And when I compact then it's going to squeeze all of that conversation which admittedly isn't very much into a much smaller space. And this in diagram terms kind of looks like this where you take all of the information from the session and you essentially create a history out of it, a written record of what happened.

And devs love compacting for some reason, but I hate it. I much prefer my AI to behave like the guy from Momento because this state is always the same. Always the same. Every time you do it, you clear and you go back to the beginning. And so if you're able to do that and you're able to optimize for that, then you're in a great spot. So that's kind of the two things I want you to think about with LLM, the two constraints that we're working with. They have a smart zone and a dumb zone. And they're like the guy from Momento.

The uh the sort of silent idea that I'm talking against here, that I'm arguing against is the specs to code movement. Has anyone heard of the specs to code movement? ... what it is is people say, okay, you can write a program or you want to build an app. The best way to build that app is to take some specifications. So to write some sort of like document and then turn that document into code. ... if there's something wrong with the resulting code. You don't look at the code, you look back at the specs, you change the specs and you sort of just keep going like this. This is kind of like vibe coding by another name where you're essentially ignoring the code. You don't need to worry about the code. You just sort of keep editing the specs and eventually you just keep going. And I tried this. I really tried it and it sucks. It doesn't work because you need to keep a handle on the code. You need to understand what's in it. You need to shape it because the code is your battleground.

...we're going to start by using a skill which is very close to my heart. It's the grill me skill. And this grill me skill is wonderfully small, wonderfully tiny. And it helps prevent one of I think the main issues when you're working with an AI, which is misalignment. ... I'm invoking the grill me skill. And what I'm going to do is I'm going to say grill me and I'm going to pass in the client brief. ... this is inside the repo so you can check it out. It's extremely short. Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies one by one. For each question, provide your recommended answer. Ask the questions one at a time. uh blah blah blah. What this does, and what I noticed when I was working with AI, especially in plan mode actually, is it would really eagerly try to produce a plan for me. It would say, "Okay, I think I've got enough. I'm just goof plan."

And what I found was that I was really trying to find the words for this for for what I wanted instead of that. And Frederick P. Brooks in the design of design he has a great quote uh talking about the design concept when you're working on something new with someone when you're uh all trying to build something together then there's this shared idea that's shared between all participants and that is the design concept and that's what I realized I needed with Claude I needed I needed to reach a shared understanding I didn't need an asset I didn't need a plan I needed to be on the same wavelength as the AI as my agent. And this is an extremely effective way of doing it. ... So, should points be retroactive? There are existing lessons progress records. We're completing out timestamps. This is a really nasty question, right? Should we actually go back and backfill all of the lesson progress events? This is a kind of question that you need to be aligned on if you're going to fulfill the feature properly. This is not something I considered and Sarah Chen certainly didn't consider. ... And this grill me skill this can last a long time. This can I've had it ask me 40 questions. I've had it ask me 80 questions. I've had some people it asks a hundred questions to literally you're sat there for an hour chatting to the AI. And what you end up with is essentially this conversation history that works really nicely and works really nicely as an asset of the design concept that you're creating.

So at this point now I have used 25k tokens and all of that or loads of that stuff is gold. I want to keep that around. I've I've got 25k great tokens there. And what I want to do is kind of summarize it in some kind of destination document. So this is um the next exercise where we're going to uh we're going to write a product requirements document. And the product requirements document or the PRD is essentially that's its function. It's the destination document. And it sort of doesn't matter what shape it is. I've got a shape that I prefer and that I quite like, but you can just choose your own shape or whatever your company uses. And all we're really doing is summarizing the design concept that we have so far.

... Should I be reviewing this document? Raise your hand if you think I should be reviewing the document. Yeah, I don't I don't look at these. The reason I don't look at these is because what am I testing at this point? ... I know that LLMs are great at summarization because they are they're really good at summarization. I have reached the same wavelength as the LLM, right? Using the grill me skill, we have a shared design concept. So if I have a shared design concept, all I'm doing is I'm just essentially checking the LLM's ability to summarize. So I don't tend to read these.

...we have our destination. ... How do we split it so that we don't put things into the dump zone? In other words, we have our number four. How do we split it into this kind of multi-phase plan? Well, probably what you would do at this point is you would say, "Okay, Claude, give me a multi-phase plan that gets me to this destination." Right? That sort of makes sense. This is what we've been doing before, but I have um a sort of better way of doing it now, which is that I like creating a canban board out of this. ... A camon board is essentially just a set of tickets that you put on the wall that have blocking relationships to each other. ... It has proposed that we split this setup into um five different tasks. Here we have the first one which is the schema and the gamification service. ... This is blocked by nothing. ... This one here is blocked by all of the tasks.

... this means that you can start to parallelize. You can start to get agents working at the same time on these tasks because yeah, this one needs to be done first and then these two can be grabbed at the same time by independent agents. ... this allows you um to turn those plans into optimally kind of like into directed asyclic graphs essentially where you just are able to um essentially have three phases here where you have phase one... Then phase two... And then phase three... That's why I prefer a canon board set up like this to a sequential plan because a sequential plan can really only be picked up by one agent.

There's a really, really important technique when you're kind of figuring out what the shape of this journey should look like. And it sort of comes to this very classic idea uh which comes from pragmatic programmer called tracer bullets or vertical slices.

Systems have layers, right? There are layers in your system. ... You might have a database, you might have an API, you might have a front end... Well, what I noticed is that AI loves to code horizontally. So it loves to code layer by layer. So in other words, in phase one, it will do all of the database stuff... Then it will go into phase two and do all of the API stuff. Then it will add the front end on top of that. Does can anyone tell me what's wrong with that picture? ... You don't get feedback on your work until you've really started or completed phase three. So what you really need to do is you're not until you get to phase three, you're not actually testing that all the layers work together.

You haven't got an integrated system that you can test against. And so instead you need to think about vertical layers. You need to think about thin slices of functionality that cross all of the layers that you need to. And this is a much better way to work, much better way for the AI to work too because it means at the end of phase one or during phase one, it can get feedback on its entire flow. So what this means to me is inside the PRD to issues skill up here I have got break a PRD into independently grabbable issues using vertical slices traceable...

...a traceable bullet by the way is... when you're like an anti-aircraft gunner... if you're just shooting normal bullets, you have no idea what you're firing at, right? ... Tracer bullets is they attach a tiny bit of phosphoresence or phosphor or something to make it glow as it goes. So, this means that every sixth bullet or something, you actually see a line in the sky. So, you have feedback on where you're aiming. So this is what this is the idea here is that we increase our level of feedback and we get near instant feedback on what we're building because without that the AI is kind of coding blind until it reaches the later phases.

So what I see here is that even though I've I've told it to do vertical slices, it's proposing to create the gamification service first on its own. That's just one slice there. And that to me feels like a horizontal slice. ... I want to see the schema changes... I want to see some new service being created and I want a minimal representation of that on the front end. ... So I'm going to give the AI a rollicking. ... the first slice is too horizontal. ... award points for lesson completion visible on dashboard. Yes, that's a beautiful vertical slice...

And it's at this point that the human leaves the loop. ... We can think of this as kind of like the day shift and the night shift. This is the day shift for the human, right? Planning everything, getting all the uh all the stuff ready and then once we kick it over to the night shift, the AI can just work AFK. But what does that look like?

...running your AFK agent. Now I've called this uh Ralph really because it is a it is essentially a Ralph loop and this prompt here I want to walk through this really closely. ... You're going to work on the AFK issues only. ... If all AFK tasks are complete, output this no more tasks thing. And then the next thing, pick the next task. So what we're doing here is we're essentially running a backlog or curating a backlog that our AFK agent is going to pick up. ... So, it's prioritizing critical bug fixes, development infrastructure, then traceable bullets, then polishing quick wins and refactors. ... we explore the repo, use TDD to complete the task.

Let's talk about TDD... TDD I found is absolutely essential for getting the most out of agents. Uh raise your hand if uh you know what TDD is. Cool. Okay. TDD is testdriven development. What it's essentially doing is it's doing a something called red green refactor. ... it's writing a failing test first. So it's saying, okay, I've broken down the idea of what I'm doing and I'm just going to write a single test that fails and then I need to make the implementation pass. I have found that first of all, this adds tests to the codebase and this this tends to add good tests to the codebase. ... I found that uh raise your hand if you've ever had AI write bad tests. Yeah, it tends to try to cheat at the tests because it's sort of doing it in layers. ... And using this technique, it generally is a lot harder to cheat because it's sort of instrumenting the code before it's then writing the code. So I find that TDD is so so good...

Now TDD is obviously really important and it's really important because these feedback loops are essential to AI essential to get AI to produce anything reasonable because without this AI is totally coding blind right you have to have to um if if your codebase doesn't have feedback loops you're never ever ever going to get decent AI decent output out of AI and often what you'll find is that the quality of your feedback back loops influences how good your AI can code. Essentially, that is the ceiling. So, if you're getting bad outputs from your AI, you often need to increase the quality of your feedback loops.

So let's go with um bad code first. In his book um the philosophy of software design, John Alistster talks about the ideal type of module. And let's imagine that you have a codebase that looks like this. Each of these uh blocks here are individual files. And these files export things from them. ... if these files are small and they don't kind of ex like export many things, then John would call these shallow modules essentially where they're not very um they kind of look like uh this. ... Now this is hard for the AI to navigate because it doesn't really understand the dependencies between everything. ... And it's then also hard to test this as well because where do you draw your test boundaries here?

... a good codebase looks like ... this where you have what John Asterhout calls deep modules. Modules that have a little interface on there that expose a small simple interface that have a lot of functionality inside them. Now what this means is that these are easy to test because you just let's say that there's a dependency between this one and this one. ... what you do is you just wrap a big test boundary around that one module around this one up here. And you're going to catch a lot of good stuff because there's lots of functionality that you're testing and really the caller, the person calling the module is going to have a simple interface to work from.

... raise your hands if you feel like you know your codebase less well than you used to. Yeah. This is a real thing. um because we're moving fast, because we're delegating more things, we end up losing a sense of our codebase. And if we lose the sense of our codebase, we're not going to be able to improve it. ... But then how do we make it so that we can move fast while still keeping enough space in our brains? I think that this is a way to do it because what you're doing here is not only are you thinking about creating big shapes in your codebase, big services.

What I think you should do is design the interface for these modules, but then delegate the implementation. In other words, these modules can become like gray boxes where you just need to know the shape of them. You need to know what they do and sort of how they behave, but you can delegate the implementation of those modules. I found this is really nice. I don't necessarily need to co-review everything inside that module. I don't necessarily need to know everything of what it's doing. I just need to know that it behaves a certain way under certain conditions and that it does its thing. So, it's kind of like, okay, I've got a big overview of my codebase and I understand kind of the shapes inside it, understand what the interfaces all do, but I can delegate what's inside. I found that has been a really nice way to retain my sense of the codebase while preserving my sanity.

And so you might ask, how do I take a codebase that looks like this and then turn it into a codebase that looks like this? How do I deepen the modules? Well, we have hopefully it's in here. Pretty sure it is. We have a skill and that skill is called improve codebase architecture.