Welcome back to How I AI. I'm Clarvo, product leader and AI obsessive here on a mission to help you build better with these new tools. Today we have a very special mini episode because Anthropic just dropped Opus 4.8, their latest state-of-the-art coding model. And I got a few hours of early access. and I'm here to share my very early thoughts about where this model is intended to perform well, where it did a great job and totally impress me, and where there's still a little bit further to go.

Anthropic is shipping opus 4.8. It is supposed to be their step change model for agents. And there's a couple things they've called out that this model does particularly well. It's supposed to be more honest, a less design flop, longer horizon autonomy on longunning tasks, and enterprise ready. So, it means it follows its instructions. And they're saying that Swebench Pro, they're hitting 69.2% which is almost five points higher than Opus 4.7, almost 10 points higher than GPT 5.5, and 15 points higher than Gemini 3.1. Now, this model is not cheap. It's $5 per input tokens and $25 per million output tokens. And then same as 4.7, effort defaults to high and fast mode can be a lot faster.

Surprise surprise lol it's a good coding model in that when I opened up claude code and asked it to do a fairly complex oneshot brand new surface area task it did a pretty good job. So I asked build a prototyping capability in chat pod. And I gave it some architecture decisions I wanted to make, what platforms I wanted to use, how I wanted it to function. It went through plan and then it autonomously coded for I would say about 20 minutes and shipped it. And when I pushed this live to my preview branch, it worked. Um, and so I would say from a oneshot feature, it did quite a good job.

Where it failed was this last 10%. And this is really going to be my theme of this episode. It does really, really well until it doesn't do well. And I found it did not do well consistently over time with the same types of trouble. So, what it nailed here is it did take the spec, it planned the work, it shipped the feature, but then as soon as I got it live and started trying to take it to the next level, it really struggled and started to ship bugs. And even more than its inability to finish that last 10% when it was bug hunting, it hallucinated. And I am going to tell you I have not seen a straight up hallucination in a very long time. But over my experience early testing Opus 4.8 both on business use cases as well as coding use cases it 100% made up things based on hypothesis not data.

Basically what I saw is when I pointed it at existing code, it also struggled to sort of insert itself and understand the edges of where it was supposed to work. I had a couple branches in flight that I needed to rebase because we had shipped a big underlying PR and it kind of messed up the state of the code. And so I asked Opus 4.8 to rebase and check these branches for code. I had to do cycle after cycle of rebase and fixes because it was continuing to ship really edgecase bugs into the code. It struggled to understand the elevation at which it should be operating.

The other thing I reflected on when I was coding with Opus 48, is it just wasn't ambitious enough. It gave me this awesome prompt, which was build a game, then play it yourself by watching the screen and tweaking the difficulty until it's fun for a 9-year-old. Amazing. This is state-of-the-art coding agent. It's going to cook. Let me show you what it actually shipped. It shipped this, which is like fine, of course, magic, but not pushing the edges of a gentic coating. It's not 10x agentic coding blow my mind impressive. I kept saying more more do better do better and it just wasn't as ambitious as I've seen other models be.

Opus 47 was very numbersacred. You can see this table here. It was very structured and and rooted in real data. While both of these exercises did have access to the same data, Opus 48 had a harder time discovering the relevant data and it overrotated on small data points and took them as truth as opposed to what I experienced Opus 47 doing, which is it zoomed out a lot more and put everything in context. 4.7 very anchored in specifics very good strategy and 4.8 was incredibly handwavy and in fact with Opus 48 it gave me a road map and then I said we have all this. Did you search through GitHub? No, I didn't actually search GitHub. No, I didn't actually look up that data.

It's just overtuned and has kind of narrow vision. So, it's smart, it's fast, it's efficient, but it's overly confident absent true validation. It really latches on to specific data points, specific code points. It draws conclusions for them and then says this must be truth. And so it sort of misses the forest for trees both in coding and in strategy.

The ergonomics were very nice. Now I got early access. Who knows what the production latency is. But with fast mode, I anticipate you'll have this fast experience. I would use it for green field prototypes. It's really impressive on a oneshot. I think its design is better. It got rid of the italics emphasis words which were driving me crazy from quad design. It's good at tool use. It's fast. It's not annoying.

Um, where I would test it is with existing code bases in branches with real edge case with strategy work that requires you to think about numbers. I would just double check where it's really confident because my experience was its confidence was not rooted in fact. In cloud code, you now have dynamic workflows which can let you spin off hundreds of parallel sub aents. And in cloud.ai and co-work, you now can set effort control from low to max. I would say it's a good model. It's not the most amazing model. It didn't blow my mind. It has some quirks to it.