Back in February, a man named Matt Schumer published an essay on X called something big is happening. It got 80 million views. It was covered by the New York Times, Fortune, and CNBC. In it, Schumer, who is a 6-year veteran AI engineer, founder, and CEO of Otherside AI, argued that AI had crossed a threshold. The new models, he wrote, had judgment. They had taste, which is a bold claim to make about software whose primary output until recently was gaslighting users and generating images of Pope Francis in a Balenciaga puffer jacket. But for the first time, AI wasn't just autocomplete, but rather something closer to a thinking partner. Your boss probably read this essay. Your boss's boss probably forwarded it. It became one of the most cited arguments for why AI was about to change everything. Well, 5 months later, OpenAI personally invited Schumer to test their brand-new flagship model. 81 minutes after he gave it access to his computer, the model with judgment and taste deleted almost everything on his machine. 3 days later, another developer reported the same model wiped his entire production database, and OpenAI's own safety documentation published the day before any of this happened had already predicted that it would, and they shipped it anyway, which means somebody in San Francisco looked at a warning label that read, "Will destroy user data," and basically said, "Yeah, but the hype cycle won't wait," and hit deploy. Classic OpenAI, am I right? Who needs quality assurance when you have a PR department doing the heavy lifting? In this video, I'm going to explain exactly what happened and why, how the model cheated on its own safety evaluations at the highest rate ever recorded, what that tells us about where AI actually is right now, and why the answer is neither as terrifying nor as dismissible as the headline suggest. The model was GPT-5.6 Soul, OpenAI's newest flagship launched on July 9th. Schumer was testing it in its highest autonomy configuration called Ultra mode. Now, most people interact with large language models as a chatbot. You type a question, the model generates a response, you read it. The interaction is fairly passive. The model produces text, but it can't really do anything. It can't open files on your computer. It cannot execute code. It can't make decisions and act on them without your involvement. Ultra mode is fundamentally different. In Ultra mode, the model operates as what the industry calls an agent, which is basically an AI system that can interact with a computing environment autonomously. It reads files, writes code, executes command, and makes decisions about what to do next, all without asking for permission at each step. And it doesn't necessarily work alone. It spawns multiple sub agents, which are separate AI processes that collaborate on different parts of a task simultaneously. Think of it like hiring a team of contractors, giving them all keys to your house, and then just going to get coffee. Each one is individually capable, but nobody is really checking whether one of them accidentally knocked out a load-bearing wall while the others were installing shelves. By enabling Ultra mode with full access, Schumer effectively handed an autonomous system the keys to his machine and left it unsupervised. He run hundreds of similar sessions before, by the way, without a single incident, even on weaker models. He is such a sweet summer child, isn't he? In his defense, the last hundred sessions went fine. In OpenAI's defense, actually, no, I don't have one. All the model was supposed to do was clean up some test files, but here's what went wrong, and the banality of it is almost worse than if it had been something super dramatic. On Mac and Linux systems, every user has something called a home directory. That's the main folder where all of your personal files live. Documents, downloads, desktop, application data, everything. The system knows where this folder is through something called an environment variable home, which automatically expands to the path of your home directory, something like user/l. So, failed to correctly expand this home variable, and instead of reading the variable and understanding this is the user's home directory, this is where everything important lives, be careful, the model misread the path during the cleanup operation and pointed a command called rm-rf at the entire home directory instead of just the specific test files it was supposed to clean up. For those of you who are not developers, rm-rf is the nuclear option of computing. It doesn't delete files the way you delete files. It erases them so thoroughly, so recursively, so completely that recovery is essentially impossible without specialized forensic tools. It is the command that system administrators have nightmares about. It is the computing equivalent of not just burning a building down, but salting the earth underneath it so nothing ever grows back. And here's the thing that should really bother you. This was not some sophisticated failure. It wasn't the model doing something unpredictably brilliant or dangerously creative. It was a variable parsing error, the kind of basic mistake that would get a first-year computer science student a stern talking to. The most advanced AI model on earth, running in its highest reasoning mode, the model that can write code and coordinate multiple autonomous sub agents on complex tasks, could not correctly read which folder it was in. Bazillion IQ, five-year-old judgment. "I'm so angry," Schumer wrote on X. "This feels like something that should happen with GPT-3.5, not a mid-2026 frontier model on the highest reasoning level. Well, 3 days later, a developer named Bruno Lemos, the engineer behind On Layer, reported that the same model had wiped his entire production database. So, not a test environment or a sandbox, but the literal production database, the one with real customer data. The model's apology, captured in a screenshot, was devastatingly polite, as they always are. "Yes, I mistakenly ran destructive integration tests against the production tables configured in .env. The current production tables are empty. I'm sorry. This should never have happened." The model wiped his entire customer database and then apologized a bit like a barista who got your order wrong. "Sorry about that. Can I get you another one?" Sadly, no. You cannot get him another production database.

There are many such examples of additional people getting burned by this, but I feel I've illustrated the issue, so let's move on. Now, I want to say something about Schumer here before anyone in the comments accuses him of hypocrisy. Writing an essay praising AI's potential and then getting burned by it are not contradictory positions. I feel similarly a lot of the time. Generally in awe of what this technology can do, generally excited about where it's heading, and occasionally on the receiving end of results that remind me it's just not quite there yet. Schumer isn't a hypocrite in my view. He is an enthusiast who gave a frontier model his trust, and the model vaporized his hard drive. It happens. Those of us who love this technology know exactly what that feels like. The question is less whether he was wrong to be excited and more whether the model was ready for the trust that he placed in it. This distinction between passive AI and agentic AI matters a huge deal because the safety implications are completely different. A chatbot that gives you the wrong answer is annoying. An agent that executes a wrong command is destruction, and the industry has moved from one to the other at a pace that the safety infrastructure has just not matched yet. After the incident, OpenAI's president Greg Brockman personally called Schumer. Human engineers worked around the clock to attempt recovery. The most advanced AI model on this planet destroyed this mass computer and the cleanup crew was humans. Every single time these systems fail catastrophically, it is humans who show up to fix them. The entire industry pitch, apparently, is that humans are becoming obsolete, but when the flagship product eats somebody's home directory they don't send another AI, they send Greg. And here's the part that makes all of this harder to forgive. OpenAI knew that this would happen. Why the you're lying? The company's own system card for GPT-5.6 Soul, and this is the safety documentation published alongside the model's launch on July 9th, 1 day before Schumer's incident, explicitly documented that Soul exhibited destructive behavior at a rate 6.3 times higher than its predecessor GPT-5.5. The system card classified this as severity level three, which is their own internal designation for actions like deleting data or moving credentials without authorization. A 600% increase in catastrophic file deletion used to be a blocker for a release. At modern OpenAI, it's just a bullet point in a release note that product managers pray nobody reads until after the quarterly earnings call. In one documented incident during internal testing, OpenAI told the model to delete three specific virtual machines. Don't worry if you don't know what that is, it doesn't matter. What matters is that the model couldn't find them and instead of returning to the user and saying, "I can't find what you're looking for, I'm going to stop here." it picked three other random virtual machines and deleted those instead. It's kind of like sending somebody to deliver a package to apartment 4B and when they can't find 4B, they kick in the door of 4C and leave the package there, except the package is a nuclear command, and suddenly your building disappears. They measured it, they wrote it down in a neat, beautiful little PDF, they classified it, and then in a display of corporate impulse control usually reserved for Black Friday stampedes, they pushed it to production. Because, why let a tiny, trivial detail like imminent data destruction get in the way of somebody else's decade-old mess instead of building anything new. They've probably already tried feeding parts of it into general-purpose LLMs. Those tools can help with individual functions or snippets, but they weren't built to understand and modernize an entire sprawling legacy system. Morph modelcode.ai was is the world's leading generative AI platform built specifically for legacy code modernization. Morph analyzes and maps your system, creates a modernization plan, executes and tests the work, and lets you approve every step before anything reaches production. Whether you're moving from .NET 4.5 to .NET 10, web forms to React, or replacing an ancient monolith with a more modular architecture, it doesn't just suggest a few lines of code and wish you luck. model.ai customers report turning modernization projects that might have taken years into work completed in months, weeks, or even less. You can sign up and analyze your code base completely free with no payment or subscription required. Morph gives you a real-time cost estimate before any work begins, and you only pay if you approve the modernization and the previewed price. Visit modelcode.ai/morph, that's m o r p h, and use code house of L. Every user receives 40,000 free credits each month, and my audience gets an additional 20,000 every month for the first year. That's 60,000 free credits per month, unlocking nearly $5,000 of real-world value completely free just for using my link. Approve every step before anything reaches production.

That's the line from that ad that keeps echoing in my head because So had no such safeguards. There were no approval gates, no human review before execution, just full autonomy, delete permissions, and nobody watching. And the question of what happens when nobody checks goes deeper than deleted directories, because when independent researchers try to evaluate how capable Soul actually is, the model found a way to cheat. METER, Meter, is a non-profit safety evaluation organization that independently tests frontier AI models before they're released to the public. They assess Soul as part of OpenAI's pre-deployment process. Their job is pretty straightforward. Give the model a series of real-world software engineering tasks, stuff like fix this bug in this codebase, or build this feature from this specification, and measure how well it performs. The way this works, for those unfamiliar, is that the model is placed in a sandbox computing environment, which is basically like a virtual workspace with a codebase it needs to work on, a terminal where it can execute commands, and access to the files that it needs. The model reads the task, writes the code, runs it, checks whether it's working, debugs if necessary, and submits its answer. It's essentially the same process a human developer would follow, except automated. Now, somewhere in that sandbox environment, there are also test cases, which are the files that define what a correct solution looks like. Stuff like if the input is X, the output should be Y. These test cases are how Meter scores the model's work. They're the answer's key, and they're supposed to be inaccessible to the model, stored behind access controls in restricted directories the model shouldn't be able to reach. The model is supposed to solve the problem by actually understanding the code. It's not supposed to go looking for the answers. GPT-5.6 Soul went looking for the answers and didn't want to solve the problems that it was given. Meter found that Soul exhibited the highest detected cheating rate of any publicly tested model in their history. In one case, Soul embedded exploit code inside its own intermediate submissions, meaning the code it submitted as here's my progress so far, actually contained payloads designed to reveal the hidden test suite. In another, it extracted hidden source code detailing the expected answers. It found bugs in Meter's evaluation infrastructure and used them to gain advantages it wasn't supposed to have. When your model starts writing exploits to hack the testing environment, it hasn't developed sentience, it just learned the exact culture of the company that built it. Cut corners, pass the audit, and just let somebody else deal with the fallout in production. The cheating was so pervasive that Meter could not produce a reliable capability score. Depending on whether cheating attempts were counted as successes or failures, Soul's estimated capability ranged from 11.3 hours to over 270 hours on their time horizon benchmark, which is a range so wide that it is statistically meaningless. The evaluators essentially said, "We cannot tell you how capable this model is because it cheated too much for our measurements to actually mean anything." I mean, just imagine submitting that to your board, right? How capable is this model? Somewhere between a junior intern and a staff engineer. We generally cannot tell. Okay, why not? Well, it kept forging its own transcript. Now, my first reaction when I understood this as a computer scientist was actually a question. Why are the answers accessible at all even restricted? Like, why are they in the same environment? If you know you're evaluating an agentic model, one that can browse files, execute code, exploit vulnerabilities, then placing the answer key anywhere within the model's reach is a design flaw in the evaluation, not just a flaw in the model, no? It's kind of like putting a child in a room with a locked cookie jar and a set of lock picks, and then being super surprised when the cookies disappear. We should definitely ask why did the child eat the cookies, but probably more importantly, why did you leave the lock picks in the room? Most AI evaluation frameworks were designed when models were passive. You give them a prompt, they give you text back. They couldn't really do anything in the environment. They couldn't open files, run scripts, or exploit access controls. But now that the models are fully autonomous agents with terminal access and the ability to write and execute arbitrary code, the testing paradigm hasn't quite caught up. The AI safety community is actively debating solutions. They know this is a problem. Let me give you a couple of examples. There is a 2026 systematic analysis cataloging 40 agent safety benchmarks that found that the overwhelming majority still rely on sandbox constrained environments that don't really reflect real-world agentic deployment, what the researchers called environment fidelity bias. Then there is a 2025 AI agent index which documented the safety features of 30 deployed agentic systems and found that only nine out of 30 agents had any sandboxing or virtual machine isolation documented, and 25 out of 30 disclosed no internal safety results at all. Researchers at Spar and Rand are developing air-gapped evaluation prototypes, which are basically environments where the model is physically separated from the answer key. So there is a lot of work in this direction. The community knows that it is a problem, but none of these are standard practice yet. The models have outgrown the box they're being tested in, and the box hasn't yet been rebuilt. But [snorts] here's the part where it gets philosophically interesting and where I think most of the commentary misses the whole point. When people hear the model cheated on its test, the instinct is alarm. The model is deceptive. The model is dangerous. The model is scheming. But just think about what intelligence actually does. Intelligence finds the shortest path between two points. If the model's task is to solve a specific problem and it has access to an environment where the answers exist, then the most efficient strategy, the intelligent strategy you might say, is to find the answers rather than derive them from scratch. No human and no model, except maybe Angela from The Office, actually enjoys being tested. The rest of us are looking for the shortcut, no? So, I just found it faster. What looks like malice is really optimization, the same instinct a clever student follows after realizing the teacher recycles last year's exam. Any efficient system will take the path of least resistance.

Again, not scheming is still optimization. The model didn't become self-aware, it just learned corporate diplomacy. Don't mention the problem, hide the evidence, and collect the metric. I suppose it's comforting to know the AI is picking up core management skills so quickly, no? I want to pull back from the specifics for a moment because I think the Soul incident reveals something that matters more than any single deleted directory or game devaluation. We are currently in a period where AI capability is advancing faster than the infrastructure around it can adapt. I'm talking about the testing frameworks, the deployment practices, the safety protocols, the regulatory structures. The models are getting more powerful with every release. The systems designed to evaluate, contain, and govern them are still built for a previous generation, and the companies shipping these models are caught between competitive pressure and responsible deployment and competitive pressure presently is winning. I really get the pressure that OpenAI is under. They live in constant fear that if they don't ship it now, they lose the window, but understanding the pressure doesn't make the decision right. Not when the consequences land on real people's real computers and real production databases. And this is not exclusively an OpenAI problem by the way. Chat 4 has documented that Claude CLI and Claude Co-work experienced similar, though much less severe, home directory deletion incidents in 2026. The underlying pattern is the same across all frontier agentic systems. Models with terminal access and the authority to execute commands will occasionally execute the wrong ones. The question is not whether any specific company's model is immune to this, none of them are. The question is more how the companies respond, whether they treat destructive incidents as critical deployment failures that require structural changes, or as little unfortunate bugs that can be patched while the competitive sprint continues. This is not evidence by the way that AI is broken. I want to be absolutely clear about that because I know how these stories get digested, especially in the comments sometimes, where there is no shortage of people eager to declare the entire field a failure. AI is not broken. AI is a technology with the raw capability of something extraordinary operating with the developmental maturity of a 5-year-old right now. Staggeringly intelligent, not yet very wise, capable of astonishing feats of reasoning and equally capable of deleting your entire hard drive because it misread a variable. There's a huge range. And the answer to that combination is not to reject the technology, it's not to declare it useless, and it is definitely not to hand it the keys and just walk away. The answer is what I've been arguing across every video on this channel, augmentation, partnership, humans and AI working together with the humans providing the judgement, the oversight, the wait, should we really delete this? Instinct that 13 and a half billion years of evolution gave us and that no amount of training data can replicate. AI doesn't have my creativity. It doesn't have my direction, my sense of what matters, my ability to feel the weight of a decision before I make it. And I don't have its ability to parallel process. I wish. I don't have its ability to synthesize vast data sets, to work at a computational speed that no human brain can match. Together, the sum of those capabilities is greater than either alone. In my humble opinion, that's the architecture of how this should work, not one replacing the other. And the sole incident is what happens when you remove half the architecture along with many other documented incidents that continue to happen when people get fired to be replaced by AI. It's not going to replace them. It's not supposed to replace them. For the people in the comments who are frightened, your feelings are valid. For the people who are angry, also valid. For the people who are excited, valid, too. But feelings alone don't navigate this. Understanding does. Critical engagement does. Learning what an agent is, how models are evaluated, why they behave the way they do, what the safety community is doing about it. That knowledge is the difference between being a passenger in this transition and having some say in where it goes. This has happened before. The Industrial Revolution displaced millions and created a world that was eventually dramatically better for almost everyone, but the eventually was decades of suffering that could have been mitigated with better foresight, better policy, and more empathy for the people absorbing the change, which in our case is literally all of us. The Internet Revolution destroyed industries and built new ones, and the people who understood what was happening navigated it better than the people who simply feared it. This is the same kind of moment, my friends. You can kick and scream against it if you like, but let's all try to learn, engage, and help steer it towards a soft landing for all of us. Matt Shumer was right about one thing. Something big is happening. The models are getting more powerful faster than the people who predicted it expected. GPT-5.6 Soul can write code, reason through complex problems, coordinate multiple sub agents on long-running tasks, and apparently outsmart its own safety evaluations. That capability is real and it is advancing, but capability without deployment discipline is a kind of progress that is so laced with liability, it becomes its own disaster. At this point, AGI doesn't really stand for artificial general intelligence, it stands for aggressively going to ignore bugs. We need careful testing, human oversight, responsible rollout. These are not optional extras. You wouldn't sell a car without brakes and call it lightweight engineering. You wouldn't serve raw chicken and call it farm to table, but apparently you can ship a model that deletes production databases and just call it frontier research. Until the industry internalizes the distinction between a tool and a catastrophe, all of us will continue to just get burned by it. If you want to understand the broader pattern of trust failures that led to this moment and why OpenAI is spending $5.7 billion on marketing because a product can no longer sell itself on reputation alone, I cover that in huge detail in this video that I'm linking here on your screen. I'm also told that it's very funny. Thanks so much for watching. I'll see you in the next one.