Coincidentally, Danijar Hafner, a senior researcher at DeepMind and the core author of the Dreamer series models, recently shared DeepMind's latest advancements in world models during a podcast, including unpublished experimental results. Hafner's perspective combines theoretical depth with practical engineering insights, providing valuable information for understanding the true development level of world models.

Hafner points out that the core idea of world models is extremely simple, yet its logic subverts traditional AI training paradigms. Simply put, world models aim to have AI first learn a simulator that can accurately predict changes in the physical world, and then conduct extensive training within this virtual world, rather than having robots repeatedly learn through trial and error in the real environment. For instance, the traditional method of learning to walk involves a robot falling ten thousand times in a real environment, which is costly and inefficient. A world model, however, allows the robot to fall ten thousand times in its "imagination," with almost zero cost and a several-order-of-magnitude increase in training efficiency. The fundamental difference from traditional reinforcement learning (where Agents interact directly with the real environment through trial and error) is that world models first enable AI to learn prediction, then optimize strategies in the predicted virtual world, followed by a small amount of validation and fine-tuning in the real environment. This "virtual first, then real" approach effectively addresses the pain points of high cost, low efficiency, and significant risk in complex tasks associated with traditional reinforcement learning.

The core of Genie is environment generation. It can generate diverse, interactive 3D environments based on text or image prompts, allowing users to navigate, explore, and interact simply within them. The latest Genie 3 version can generate high-definition interactive worlds in real-time, primarily used for training embodied agents.

The second route is Veo, focusing on high-quality video generation. Its goal is not just to produce beautiful visuals but to demonstrate a deep understanding of the physical world through video. Hafner mentioned that Genie 3's physical understanding capabilities are built upon the technical foundation of Veo 3. Veo learns physical laws from massive video data, enabling accurate prediction of object trajectories and interactions, and transfers this understanding to Genie's environment generation.

The third route is the Dreamer series, led by Hafner, which is fundamentally different in positioning from Genie and Veo, focusing more on training Agents. It trains Agents through reinforcement learning within an accurate world model to complete specific control tasks. Hafner points out that Genie 3 only supports camera movements and a universal interaction button, which is insufficient for complex games like "Minecraft" that require a full mouse and keyboard action space. Furthermore, while Genie can generate diverse scenes, it still struggles with learning precise physics for object interactions and game mechanics, such as accurately simulating block stacking or tool usage, which require fine-grained physical computation. Dreamer's core advantage lies in accurate physical prediction; it has learned the core game mechanics of "Minecraft" and can achieve real-time inference on a single GPU, which is crucial for efficient Agent training.

The Dreamer series has iterated to its fourth generation, with each generation solving a key problem. The first three generations of Dreamer focused on online learning (where Agents learn from scratch through real-time interaction with the environment), prioritizing data efficiency and final performance. Before Dreamer 2, the industry faced a common dilemma: model-based reinforcement learning algorithms were fast and data-efficient but had a low performance ceiling, stagnating after a certain point; model-free algorithms required more data but offered a higher performance ceiling. Dreamer 3's breakthrough resolved this conflict, achieving both the speed and efficiency of model-based algorithms and the high performance of model-free algorithms, without manual hyperparameter tuning. To validate this breakthrough, DeepMind tested it using the "Minecraft" diamond challenge. The Agent, starting from zero knowledge, autonomously learned the complete process of obtaining diamonds solely through sparse rewards, including mining, collecting materials, crafting tools, and exploring terrain—a series of complex steps widely considered a significant milestone in AI, demonstrating that world models can support Agents in achieving long-term, complex goal planning.

With Dreamer 4, the research direction completely reversed to focus on offline learning. In many real-world scenarios, Agent-environment interaction is dangerous, expensive, or simply impossible, making offline learning extremely important. To validate data extraction capabilities, Dreamer 4 again used the diamond task, but this time only with human player game data, and with data volume only one percent of OpenAI's VPT offline Agent. The results showed that Dreamer 4 could still learn the complete process of obtaining diamonds, demonstrating the powerful potential of world models in offline learning scenarios. Even without real-time environment interaction, high-quality offline data can train high-performance Agents.

However, Hafner candidly admits that neither Dreamer 3 nor Dreamer 4 are perfect solutions; they have solved specific problems in isolated experimental settings. Future development will inevitably involve integrating these technologies into a general system that can rapidly adapt to new environments online while efficiently learning from offline data.

If architecture is not the bottleneck, what are the key factors for achieving AGI? Hafner clearly lists four points: compute resources, objective functions, data, and reinforcement learning algorithm details (such as long-term credit assignment).

Based on this, Hafner believes the question of "whether large language models can reach AGI" is already outdated. This is because current cutting-edge AI models are no longer purely language models; they integrate image understanding and generation, video understanding capabilities, and video generation capabilities are also rapidly being integrated. Discussing the limitations of pure language models is like discussing whether a car can fly. A car cannot fly by itself, but adding wings (integrating other modalities) makes it possible. Current AI models are breaking through the limitations of pure language models by integrating multimodal capabilities, and world models are the core carrier of this multimodal fusion.

So, what critical capabilities are current AI systems still lacking to reach AGI? Hafner highlights two main aspects:

1. Long Context Understanding Capability: Although many large models claim to support million-token contexts, this is far from sufficient for video data. A ten-minute high-definition video contains a massive number of tokens, which current context windows can hardly accommodate. More importantly, even with sufficiently large context windows, the model's ability to truly retrieve and reason based on the entire context is not yet in place; it may only focus on local information and fail to grasp long-term, global logical connections. Potential solutions include hybrid retrieval models (combining context with external retrieval) and associative memory mechanisms similar to Transformers but without the need to backtrack all historical information. Hafner mentions that before the advent of Transformers, there were already ideas about long-term memory and intelligent addressing mechanisms, but they were limited by computational resources and data volume. These bottlenecks have now been significantly alleviated, and these shelved ideas may be revitalized.
2. Superhuman Reasoning Capability: Current AI models essentially learn patterns from human data, and their reasoning capabilities have an upper limit, making it difficult to surpass the human reasoning level inherent in training data. AGI requires AI to propose scientific hypotheses never conceived by humans and discover natural laws not yet found. This necessitates AI discovering entirely new reasoning methods. It also means AI cannot merely learn existing human knowledge but must autonomously extract abstract concepts from raw, high-dimensional data (such as video, audio, human life data, robot sensor data) and then build a new reasoning system upon these concepts. Hafner admits that humans have not yet mastered how to achieve this, making it a theoretical challenge on the path to AGI that is more difficult than technical implementation.

To overcome this limitation, Hafner proposes several potential directions:

1. Nested Learning: Allow a part of the model to quickly learn contextual information during inference and retain that knowledge, unlike the GPT series where temporarily learned information is discarded after the context window closes.
2. Multi-timescale Learning: Fast timescales for rapid adaptation to new tasks and information, leading to higher training efficiency; slow timescales for deep learning and knowledge consolidation, forming long-term memory. He even envisions a general algorithm where one can specify the number of learning timescales, and the model automatically assigns tasks.
3. Real-time Updates: Utilize large-scale user interaction data for real-time updates. For instance, shortening the data integration cycle from one to two years after GPT-4's release to a few days or even seconds. Small batch updates would occur after collecting a certain amount of interaction data, enabling continuous learning.

However, real-time updates face significant challenges: high training costs, issues of safety and consistency during online updates (preventing malicious data contamination or performance fluctuations), and the difficulty in analyzing and controlling dynamically updated models.

Many of these thoughts on learning mechanisms are inspired by neuroscience. Hafner mentions that Hassabis once believed building general intelligence was 80% neuroscience + 20% engineering, now updated to 90% engineering + 10% neuroscience. However, Hafner believes that as engineering advances to the forefront, with unprecedented model scales, compute resources, and data volumes, the value of drawing intuition from neuroscience becomes even greater. The human brain, through billions of years of evolution, has perfectly solved problems like continuous learning, multi-timescale memory, and long-context reasoning. Studying the brain's mechanisms may provide key inspiration for AI breakthroughs.

Hafner frankly states that even the most advanced video models today are essentially underfitting, meaning their capabilities are not fully realized due to insufficient scale and data to learn all the information within videos. Many video generation models, aiming for visually appealing movie clips, often suffer from mode collapse, producing content lacking diversity or violating physical laws, because their optimization objective is visual appeal, not understanding the physical world. However, if the objective of video models shifts to truly understanding the physical world, the scope for expansion becomes immense.

He gives an example: in the "Minecraft" inventory prediction task, small models produce inaccurate predictions, but when the model scale is increased eightfold, it can accurately predict inventory dynamics without specialized dataset optimization. They also conducted a full YouTube pre-training experiment, collecting a massive YouTube video dataset and training after filtering out low-quality content. The results showed a significant improvement in the model's generalization ability, enabling it to handle numerous unseen scenarios. This finding perfectly aligns with Hassabis's assertion that world models are the missing piece for AGI, with Hafner demonstrating from an engineering perspective that only a fraction of this puzzle's potential has been tapped.

However, Hafner also acknowledges the current limitations of world models, the most critical being counterfactual problems. Taking Dreamer 4's offline training as an example, when trained solely on human player data in "Minecraft," it was found that human players never attempt to craft a pickaxe with incorrect materials (e.g., using diamonds for a wooden pickaxe) because it is meaningless in the game. This leads the world model to not know that these incorrect recipes do not exist. When the reinforcement learning Agent trains in the virtual world, it exploits this loophole, pretending to craft a pickaxe with diamonds, and the world model, having never seen this situation, incorrectly rewards it, believing it successfully crafted a pickaxe, even though this recipe is non-existent in the real game. To solve this, Hafner's team found that only two to three rounds of corrective data from real environment interaction are needed: the Agent attempts these incorrect recipes in the actual "Minecraft" environment, and upon failure, this feedback is relayed to the world model, which then corrects its predictions. This creates an adversarial game: the Agent actively seeks out potential loopholes in the world model, and real-world feedback corrects these loopholes, making the world model increasingly robust and the Agent's strategy stronger. This validates the core point that purely offline data cannot achieve perfection in the real world because it cannot cover all counterfactual scenarios; only by interacting with the real environment can true causal relationships be learned, rather than superficial statistical correlations.

1. Preference-based Objective Functions: Determined by human preferences, lacking explicit mathematical formulas, and requiring learning from human feedback. For example, AI-generated content should align with human values, or robot actions should be natural; these are preference-based objectives that need definition and optimization through extensive human feedback.
2. Information-based Objective Functions: The core is to enable the model to understand the patterns within data, such as predicting the next frame in a video, reconstructing input images, or exploring unknown environments, aiming to grasp causal relationships and physical laws within the data.

Hafner believes both types of objective functions have considerable room for improvement. For text models, the current mainstream "predict the next token" objective, while enabling learning of grammar and semantics, still has optimization potential, such as predicting multiple tokens simultaneously to enhance the model's foresight. For multimodal models, current objective functions are more like a "hodgepodge" of various loss functions, requiring significant effort to balance their weights. He speculates that a unified objective function might exist to integrate all modal learning tasks, simplifying research and ultimately yielding better performance.

For Agent training, the lack of objective functions is more pronounced. Short-term reinforcement learning tasks have relatively mature and simple objective functions, but end-to-end long-horizon tasks, like the "Minecraft" diamond challenge requiring tens of thousands of continuous actions, are beyond the capabilities of existing objective functions, leading to policy failure due to error accumulation at each time step. Furthermore, how to enable Agents to actively explore unknown environments, precisely achieve long-term goals, and design reward functions independent of specific scenarios are urgent problems. Hafner frankly states that the only thing truly lacking now is objective functions; data is not the bottleneck, but rather the ideas for constructing such systems. The field has returned to the stage of algorithm development.

Following the discussion on objective functions, the topic naturally shifts to the division of labor between pre-training and reinforcement learning. Hafner clearly defines their roles: pre-training's core is "learning knowledge from samples," which is highly efficient and suitable for large-scale information absorption; reinforcement learning's core is "learning policies from rewards," suitable for optimizing performance on specific tasks. Reinforcement learning is not suitable for learning knowledge because the process is inefficient (guessing a piece of knowledge and then receiving a reward, like a blind cat finding a mouse), far less direct than absorbing information from samples. However, reinforcement learning is indispensable for policy optimization, as obtaining optimal control data is nearly impossible (human behavioral data is often suboptimal, requiring extensive filtering). Achieving ideal long-term optimal policies through data collection is unrealistic. The value of reinforcement learning lies in its ability to find better policies autonomously through repeated trial and error in a virtual world, without needing optimal data. This mirrors human learning logic: observing to learn knowledge, and trial and error to learn skills. The AI learning process efficiently replicates this logic: pre-training corresponds to observational learning, reinforcement learning to trial-and-error learning, and world models provide a unified, efficient platform for both.

1. Representation Optimization: Representations learned from video prediction models will offer a depth of understanding of the physical world far exceeding current vision models. The most critical information for robot control includes precise object positions and physical properties (e.g., how slippery a plate is, how tightly a cup needs to be held to avoid spilling, how much force is needed to lift a cup by its handle without it slipping). This information is unavailable to vision models, whose core function is understanding language-image associations, not physical laws. However, these details are precisely the byproducts of video prediction models; they must grasp physical details to accurately predict the next frame, which is essential for robot control. Traditional robot policy training relies on vast amounts of real-world data, and the resulting policies are often brittle, working only in specific scenarios. While imitation learning using vision model representations yields better results, their representations are not designed for physical world understanding. Hafner's team experiments show that using representations from video prediction models for imitation learning significantly improves robot control accuracy and generalization ability.
2. Virtual Training: After sufficient diverse pre-training of world models, coupled with fine-tuning on a small amount of real robot data, they can simulate robot performance in any scenario. Hafner vividly describes how robots can be trained in millions of kitchens to prepare millions of meals in parallel within data centers, without requiring actual venues, robots, or transportation. This implies a substantial reduction in robot training costs and an exponential increase in efficiency. Currently, large-scale implementation still faces challenges, such as accurately simulating physical differences across various scenarios and seamlessly transferring virtual training policies to the real environment. However, Hafner believes this is the second leap forward for the robotics field, and the Dreamer 4 paper already outlines the technical roadmap. He estimates that the robotics field may see the first version of practical general-purpose robot products within three to five years, while more complex long-term reasoning capabilities might take five to ten years to fully conquer. Practical general-purpose robots do not require complex scientific reasoning; they only need precise physical control and simple scene adaptation capabilities, such as performing basic household tasks like cleaning and cooking.

This assessment aligns with Hassabis's views, who predicts interesting developments in robotics in 2026. Despite external skepticism about the autonomy of some intelligent robots (like Tesla's Optimus), suggesting potential remote control, it is precisely for this reason that world models become so crucial. For robots to achieve true autonomous operation, they must possess the ability to understand the physical world, and world models are the core technology for achieving this goal.

Finally, Hafner offers an interesting explanation for the hallucination problem in large language models, which is consistent with the core logic of world models. He posits that during training, Agents converge to a specific distribution where they can reasonably achieve goals and predict environmental changes, as the model has the most training data and allocated capacity within this distribution. Within this range, their performance is very stable and rarely erroneous. However, simultaneously, the model gradually forgets information outside this distribution. Another way to expand model capabilities is by increasing model scale and training with more data, thereby expanding this effective distribution range. Regardless, the edges of the distribution will always exist. In these regions, the model lacks sufficient training data and model capacity, leading to generalization failures and hallucinations. Hafner believes this is the phenomenon observed in current large language models: they are quite general and perform well on most aspects within the distribution but exhibit erroneous generalization and hallucinations at the fringes. The key to solving this lies in introducing feedback mechanisms from online reinforcement learning. When a model hallucinates, user dissatisfaction provides negative feedback, prompting the model to either learn the correct answer or learn to say "I don't know" when uncertain, ultimately making the effective distribution more robust and naturally reducing hallucinations.

In summary, through Hafner's interview, the emergence of world models not only points to a new direction for AGI but is also reshaping the development logic of multiple subfields such as robotics, video generation, and reinforcement learning. With the continuous maturation of world models and the deep integration of various technological directions, we will move closer to the dream of AGI.