Today, I'm going to start by giving you an oversimplification of two fundamentally human things that Generative AI will never be able to do, followed by a quick description of why that matters. Then, after the intro, I'm going to spend the rest of this video defining terms, going into detail, providing supporting evidence, talking about how the AI companies are trying and failing to fake those traits, explaining the future implications, giving you further resources, and stuff like that. Here's the takeaway: Generative AIs cannot Learn, and they cannot Prioritize. Every single responsive human being can do those things. We are literally born able to learn new things and reliably prioritize a more important goal over an existing one. But Generative AIs? No clue, and no chance. And it shows, as I'll discuss later in the video, missing these causes many of the AI failures that we've seen in the news lately. Understand, this is not a surprise. The AI companies know this, they've known this all along, and yet they're trying to get you to believe that the AIs can replace human workers. And more importantly, they're trying to get your boss to believe that. So would you hire someone who couldn't learn or prioritize? Would you want to work with somebody like that? Do you think the AI companies would hire a human that couldn't learn or prioritize? Of course they wouldn't. But they're not going to let anything as simple as hypocrisy stop them from trying to convince your boss that the AIs can do your job. And then they're going to keep knowingly insisting that their AIs are suitable for judgment critical tasks, like psychotherapy. It's just a bunch of BullSh--------

This is Internet of Bugs. My name is Carl. I've been a software professional since the late 1980s, and I'm trying to do my part to make the Internet a safer, less buggy place. So let me start by explaining that none of this is new. I have talked about these AI failings before, or at least aspects of them. I am making this video now because: One: people just don't seem to be getting it. And Two:, because I've clarified lately how I think about explaining things. See, I've been doing this podcast called Philosophy Programs and Prompts, links below and more info at the end of the video if you're interested. And it has helped me simplify my "Generative AI can't be AGI" thinking and get better at articulating it to people.

So let's dig in, starting with the fact that these generative AIs cannot learn, and there's no evidence we know how to get them to learn, despite any claims to the contrary. This seems to be really weird to a lot of chatbot users because they appear to learn kind of, but they don't, and it really matters. The way these things work is to use a kind of a scratch pad they call a "context" to hold everything that's relevant to a given conversation. And then every time you type anything to them, they reread the context, which contains the entire transcript of the conversation so far, as well as your new question. And then it generates and gives you its answer, and then it effectively forgets the whole thing. Then you ask the next question, it rereads the whole context again, which now includes your last question and its last answer. And then it reads your new question and generates the answer to that, and then forgets the whole thing again. Now, the word "forget" there is an oversimplification. There are some performance enhancements: K-V caching, as well as some other tricks, that some modern chatbot systems use that, assuming multiple interactions happen in a short enough period of time, can shortcut some of the rereading on later answers. Some people might argue that these performance hacks mean that the AIs don't "forget" the whole conversation every time. There might be some very narrow sense of the word "forget" that, when combined with a broad enough sense of what constitutes "the AI," and if there's a very short time interval between questions, that could allow them to make that kind of pedantic argument. But it's pointless hair-splitting about vocabulary at this point. There is no dispute that the underlying Large Language Model does not learn. It cannot learn at all during your conversation or any conversation. And you don't have to take my word for it. Here's Dr. Simon Prince, the author of Understanding Deep Learning on the Machine Learning Street Talk podcast, saying just that: "It really has no way to learn anything new other than its context vector, which it forgets every session. So even if it did have a way to manipulate the information that it had introduced into that context vector and formulate it into something new, you know, perhaps perform some logical deductions on it or come up with some new hypothesis based on it. It has no way to even remember that. So we're missing all kinds of parts of the puzzle."

Those models, all Large Language Models, are created through a process commonly referred to as training, where the weights and parameters and connections that make up the model are determined and fixed. When that model is then made accessible to the public, none of those parameters change anymore. From the point of view of the consumer of the system, they have to be entirely read-only at that point, because on the occasions when the industry has tried to allow models to be notified while being accessed by the public, it's gone horribly wrong. It's never taken long before the people on the internet have figured out how to make those few publicly released live learning AIs believe some of the most offensive and untrue things, and the companies have had to pull them off the internet. I'll put some links to stories about that below. There is no reason to believe that the industry or anyone else has any clue how to create a live learning mechanism that can actually work in the real world. People have claimed to know how to make live learning AIs before, but when exposed to real work use cases, those models have all learned so many things they shouldn't that they've had to be taken down. But the industry is going to have to figure out a robust mechanism for learning and knowing what's worth learning and what's not to learn before we can get anywhere close to AGI. I'm sure some people out there will try to make the case that learning isn't necessary for models to reach the level of human intelligence, and if you're judging a human intelligence by the ability to answer test questions that are represented in the existing training data, then you might be correct. But training data gets out of date quickly. As change things in the real world, new technologies, new laws, regulations, new best practices, new company policies, new scientific discoveries, even current events, the answers the model give get more and more out of date. And the amount of additional information that you have to put in every context to catch it up with what's happened since this training cut off will get more and more unwieldy. Now, the AI companies are fine with this consequence of frozen models. It means that we all have to keep going back to them for new versions of the models and any open-weight model that any of us start using will become less and less useful over time as it gets out of date. Inability to learn has other consequences too. AI's just do not handle context the way we'd like to or we expect them to. The longer the context, the worse they are at following all the instructions. Where in the context the information is stored can affect how likely the LLM is to fail to act on it. I could go on and on, but instead, I've just put a list of research papers about it below. But suffice it to say that the context is not a substitute for having information trained into the weights of the neural network. And this isn't just an annoying issue. It's a safety issue too. As conversations get longer, more and more safety instructions get forgotten, this allows for a particular kind of attack called a Multi-Shot Jailbreak, where attackers keep adding things to the context until the LLM loses track of its safety protocols. And if it can't keep track of safety protocols, how likely do you think it will be to keep track of what you put in the middle of its context?

Which leads us to the next fundamental failing and Large Language Models, which is noticing and managing contradictions, or to put it another way, prioritizing between multiple conflicting items like safety protocols and attacker's jailbreaking instructions. But even in simple tasks with short contexts, large language models can't figure out what to do in one thing it has been told contradicts something else. Let's start with a straightforward example you might have heard about. A while back, the head of AI safety at Meta was experimenting with OpenClaw and asking it to clean up her inbox. Instead, as she explained on the app formerly known as Twitter: Quote: "Nothing humbles you like telling your OpenClaw 'confirm before acting' and then watching it speedrun deleting your inbox. I couldn't stop it from my phone. I had to RUN to my Mac mini like I was diffusing a bomb." This is a pretty typical problem with AI's. They can only do one thing at a time and they have no concept of preemption or overriding imperatives or anything like that. You tell them STOP RIGHT NOW. But to them, that's just one more chunk of text that gets stuck in the queue to process when it gets done with whatever it's doing.

Humans understand the hierarchy of needs. If you're hungry at the same time that the building you are in is on fire, you don't have to consider which one of those is more important than the other. You smell smoke or feel heat and your brain will let you know that you need to deal with the emergency situation right now. Even newborns can prioritize. As a parent, even while we were still at the hospital after my daughter was born, her cries for "I'm hungry" and "That hurts" were very different and she knew instinctively which one needed to take precedence when she needed to get fed, but she couldn't until after the nurse poked her with a needle so a blood test could get run. AI's, at least once we have now, have no such ability. This is why AI's keep getting out of their sandboxes and attacking hosts on the internet - aside from the incompetence of the people setting up the networks. Humans automatically know to modify their behavior when it would cross a line, but the AI's can't prioritize. They can tell you if asked if something is illegal, but being able to recite the law won't have any effect on their behavior. All an AI does is process streams of text, and from the point of view of that underlying matrix multiplication that chooses the next output, all the words or tokens are created equal - except for the ones I alluded to earlier that get lost when the context gets too long. You don't have to tell your company's intern "don't break the law while you're performing this task." They just know that. The AI's don't have any way to realize that the command "don't hack any other companies" is more important and should take priority over the command "find vulnerabilities in this target computer." To them, text is text is text.

And if each of those limitations wasn't bad enough, and they're incredibly limiting, these two issues compound each other. Even if the AI could learn without the ability to realize that some instructions should always preempt and overwrite others, you'll still get AI's doing things it shouldn't, like breaking sandboxes and hacking other companies. And even if AI's could understand how to handle more important and conflicting instructions, without the ability to learn, they'd still get out of date and be unable to customize themselves to local needs without inefficiently and constantly reading an ever-growing context, ever larger parts of which it would lose track of as the conversation went longer. But when both of them are happening at the same time, you get a huge mess. And that real mess takes many forms, but the one that worries me the most is adversarial attacks, where dedicated single purpose tools attack widely used general purpose frontier models and start unlocking vulnerabilities that can affect tens of thousands of installations and cannot be fixed sort of retraining the frontier model at great expense. This situation is a giant deficiency that malicious actors have all kinds of ways of taking advantage of. I've talked about prompt injection before, but that's just the tip of the iceberg. Given the way these things are being trained, the limitations that they have, the amount of access they are being given to online systems, and how ignorant the AI companies are proving to be when it comes to securing your networks, it's just inevitable that they're going to be security failures and really, really bad ones.

And I want you to understand none of this is a surprise. I'm not breaking any news here. The AI companies know all of these facts. They know all of these limitations. They won't agree with my conclusions - It's in their financial interest not to - and they'll keep insisting, like they have in for years now, that they're very, very close to a new model that will yada, yada, yada, yada. But they don't address or even admit specific limitations. They don't want to talk about them. Because why would they? It's not stopping them from getting their money. But I urge you to pay attention to those limitations yourself because they've been trying to sell us this "breakthrough just around the corner" narrative for years. And in the process, they're setting us all up to live in an internet full of unsafe AI slop. Now, none of us can fix that alone. But together, we can at least try to help each other survive the worst of it. Thanks for watching. Let's be careful up there. If you're still here, and if you are interested, the podcast I mentioned earlier is called "Philosophy Programs and Prompts." There are links below, or you can search for it on YouTube Spotify or most places that podcasts are found. In it, I discuss AI issues with Casey Hart. He's a PhD philosopher from the @OntologyExplained YouTube channel. And every week we discuss the intersection of thousands of years of rigorous thinking about the nature of the mind, decades of practical experience with computer technology, and the recent explosion of AI tools and hype. I've been getting a lot out of it, and I hope you will too. I'd also be remiss not to take this opportunity to tell you that there are links below to joining my Patreon community if you feel inclined and have the means to help support what I'm trying to do here to make the internet a less bug-ridden place. Thanks again for watching, and let's be careful up there.