Latest / UpNext AI / UpNext AI Deep Dive: World Models, Spatial Intelligence, and the Race to Teach AI Reality
Transcript
- Welcome to the UpNext AI podcast! This is a special, deep-dive episode where we look at what's up next in world models. For the past few years, the public face of AI has been the chatbot. You type into a box, and the model responds. It summarizes documents, writes code, drafts emails, answers questions, explains concepts, and increasingly acts as a layer on top of software. That phase is still unfolding. Large language models are not going away. But among researchers and investors, a different question is moving to the center: What would it take for AI to understand the world well enough to act in it? That question is driving renewed interest in what researchers call world models. The basic idea is straightforward: a useful AI system should be able to form expectations about what happens next. If a robot pushes a box, will it slide, tip over, or stay stuck? If a camera moves through a room, what should still exist outside the current frame? If an autonomous agent takes an action, can it anticipate the likely outcome before committing to it? This is a different kind of intelligence than generating plausible text. It's closer to prediction, planning, spatial understanding, and physical intuition. And right now, it's attracting serious capital. Fei-Fei Li’s World Labs recently raised $1 billion to pursue spatial intelligence — AI systems that understand and generate three-dimensional environments. Investors included AMD, Nvidia, Autodesk, Emerson Collective, Fidelity, and Sea, with Autodesk alone reportedly contributing $200 million. Former Meta Chief AI Scientist, Yann LeCun's new company, ami Labs, has raised more than $1 billion in seed funding to pursue world models from a different angle — one that explicitly challenges the idea that today’s large language models are the main path to more general AI. So today, we’re looking at the world-model bet. Why some leading AI researchers think language models are insufficient. Why video generation changed the conversation. How this connects to robotics, but isn't simply robotics with more compute. And why the investment case is both compelling and risky. The upside is large: AI that can reason about space, time, action, and consequence. The hard part is that the physical world is much less forgiving than text. PART 1 — WHY LANGUAGE MODELS MAY NOT BE ENOUGH Large language models were built around a surprisingly simple training objective: predict the next token. That objective turned out to be far more powerful than many people expected. At scale, next-token prediction produced models that can write, translate, code, summarize, classify, reason through some problems, and serve as flexible interfaces to information and software. That's the central lesson of the LLM era: prediction at scale can produce broad capability. But there is a limit to what that kind of prediction gives you. A language model can describe the world. It can infer patterns from text about the world. It can answer questions about cause and effect, at least when those relationships are represented in its training data. But it doesn't necessarily have a reliable internal model of physical consequence. Yann LeCun’s critique is that GPT-style systems are sequence models. They predict symbols from prior symbols. That is useful, but it is not the same as understanding how the world changes in response to action. For an AI system that needs to plan, the important question is different. Given the current state of the world, and a possible action, what future state should the system expect? That's what a world model is meant to provide. Fei-Fei Li sees the same gap from the perspective of vision and spatial intelligence. Her argument is that intelligence cannot be complete if it only operates through language or flat images. Humans understand the world spatially. We know objects persist when they leave our field of view. We understand distance, shape, motion, occlusion, and physical interaction. We can imagine how a room is laid out from partial information. That kind of understanding is basic for humans. For AI systems, it remains difficult. So when Li talks about spatial intelligence, she means AI that can understand, generate, and reason about three-dimensional environments — not just label images or produce convincing video. That distinction matters. A model that can say “the cup will fall” is useful. A model that can anticipate how, when, and where the cup falls is much more useful if the system is controlling a robot, designing a factory floor, training an autonomous vehicle, or generating an editable 3D scene. PART 2 — WHY VIDEO MATTERS Video changed the world-model conversation because it introduced a training signal that is much closer to lived reality than text. Text describes events. Video shows motion, continuity, perspective, and interaction. When OpenAI introduced Sora back in 2024, it described the system as part of an effort to teach AI to understand and simulate the physical world in motion. That framing was important. Sora was not presented only as a video-generation product. It was also presented as evidence that large-scale video models might learn some structure of the physical world. That claim is plausible, but it needs to be handled carefully. A model that generates realistic video has probably learned useful regularities. It may learn how people move, how animals behave, how light changes, how cameras pan, how objects tend to interact, and what kinds of scenes are visually coherent. But visual coherence is not the same as reliable physical understanding. A generated video can look right while violating physics. It can maintain the appearance of continuity while losing track of object identity. It can create plausible motion without understanding the underlying cause. That's where LeCun’s critique becomes important. He argues that predicting every future pixel is probably the wrong objective. The world contains too much irrelevant detail. If a camera pans across a room, the exact texture of every chair or the precise face of every person may be impossible to predict and unnecessary for planning. What matters is the right abstraction. His JEPA framework — Joint Embedding Predictive Architecture — is built around that idea. Instead of predicting raw sensory detail, the system learns a compact representation of an observation and then predicts in that representation space. The distinction matters. A generative video model might try to predict the next frame pixel by pixel. But the future contains too much irrelevant and unknowable detail: the exact texture of a carpet, the precise reflection on a glass, the position of every leaf in the wind. JEPA tries to avoid wasting capacity on those details. It asks the model to learn the part of the scene that is stable, meaningful, and useful for prediction. In simple terms, JEPA works by taking two related views of the world — for example, a current observation and a later or partially hidden observation — encoding them into abstract embeddings, and training the system to predict one embedding from the other. The model is not rewarded for reconstructing every pixel. It is rewarded for capturing the underlying structure that makes the future predictable. That makes JEPA closer to how science often works. Meteorology does not track every air molecule; it predicts pressure, temperature, wind, and humidity. A good robot does not need to imagine every photon in a room; it needs to know that the cup is on the table, that the table is solid, and that pushing the cup too far will make it fall. For LeCun, this is the path toward practical world models: systems that learn abstractions of reality rich enough to support planning, but not so overloaded with detail that prediction becomes impossible. And there's now early research evidence that this idea can move beyond theory. In a 2025 paper, Meta researchers introduced V-JEPA 2 — a self-supervised video model trained on more than one million hours of internet video. The striking part came after pretraining. The researchers adapted the model with less than 62 hours of unlabeled robot video, then used it as an action-conditioned world model for robot planning. In practical terms, the model could help robot arms plan simple pick-and-place tasks in two different labs, without task-specific training and without reward engineering. That's not general robotics — it's not a robot butler — but it is exactly the kind of result world-model advocates are looking for: a system that learns mostly from observation, then uses a smaller amount of embodied data to predict what actions might do in the real world. Weather forecasting offers another useful analogy. Google DeepMind’s GenCast, detailed in a Nature article, is a machine-learning weather model that produces probabilistic forecasts up to 15 days out. It's not trying to simulate every molecule in the atmosphere. It predicts higher-level atmospheric variables — wind, pressure, temperature, humidity — and generates an ensemble of possible futures. That's what a mature world model can look like in a bounded domain: not a perfect simulation of everything, but a compressed predictive model of the variables that matter. PART 3 — HOW THIS DIFFERS FROM TRADITIONAL ROBOTICS The robot example points to a natural question. If world models are supposed to help machines act in the physical world, how different is this from the robotics work that's been happening for decades? After all, companies like Boston Dynamics have always relied on models — of robot bodies, joint dynamics, balance, contact forces, terrain, motion, and control. Those models are why robots like Atlas and Spot can move through complex environments. They are the result of serious engineering across mechanical design, perception, simulation, locomotion, and planning. But traditional robotics models tend to be narrower. They're often designed around specific robots, specific tasks, and specific control problems. The new world-model push is more general. The goal is to learn representations that can apply across environments, objects, tasks, and embodiments. That might mean training on video, simulation, sensor data, robot trajectories, human demonstrations, or synthetic worlds. Boston Dynamics itself is moving in this direction. Its work with Toyota Research Institute on Large Behavior Models for Atlas is aimed at building policies that can handle longer-horizon manipulation tasks and generalize across more situations. A recent research paper called Humanoid World Models makes a related point from a smaller-lab perspective. The authors trained lightweight video models on about 100 hours of humanoid demonstrations. These models forecast future first-person observations conditioned on actions — basically, what the robot expects to see next if it moves in a particular way. The economics are interesting. The paper reports that parameter-sharing techniques reduced model size by roughly a third to a half with limited impact on quality. That points to a more grounded version of the world-model story: not one giant simulator for everything, but smaller predictive models tuned to particular bodies, tasks, and environments. So the split between “old robotics” and “new world models” is not clean. A better way to think about it is this: Traditional robotics has been very strong at modeling bodies and control. World-model research is trying to build broader predictive models of environments, actions, and consequences. The two approaches are likely to converge. The companies that win in robotics may need both: the hard-earned engineering discipline of robotics and the broader generalization capabilities of foundation models. That's also why this field is so difficult. In software, a failure can be annoying or expensive. In robotics, a failure can break equipment, damage inventory, stop a production line, or injure someone. The physical world raises the bar for reliability. PART 4 — THE INVESTMENT CASE The investment case for world models includes some of the following pieces. If AI remains mostly a software interface, language models may be enough for a large share of use cases. But if AI is going to operate in factories, warehouses, hospitals, homes, vehicles, labs, construction sites, and defense systems, it needs stronger models of space, time, and action. It needs to understand that objects persist when hidden. That liquids spill. That surfaces have friction. That tools have affordances. That the same object looks different from different angles. That actions change the state of the world. That kind of capability could matter across several markets. In robotics, it could reduce the amount of task-specific training required. In industrial automation, it could improve simulation, planning, and optimization. In design and architecture, it could turn generative AI from flat images into editable 3D environments. In gaming and entertainment, it could support more dynamic world creation. In autonomous systems, it could improve planning under uncertainty. For investors, the appeal is that world models look like a foundational layer. If they work, they could sit underneath robotics, simulation, spatial computing, design software, industrial AI, and autonomous agents. That is why World Labs and ami Labs are attracting such large rounds. The bet is on a broader AI substrate for understanding and acting in the world. PART 5 — THE RISKS The risk is that the story is ahead of the product. World models are intellectually compelling. They may also be necessary for more capable AI systems. But that doesn't mean they become a business quickly. The first challenge is technical. There's still no consensus on the right architecture, the right training objective, the right data mix, or the right evaluation methods. Should systems predict pixels, embeddings, object states, rewards, actions, or latent dynamics? How much structure should researchers build in? How much should be learned from scale? Those are open questions. The second challenge is data. Language models benefited from the internet. Text was everywhere. Spatial and embodied data are harder. High-quality video is useful, but passive video does not always reveal causality. Robot data is more valuable for action, but it's expensive to collect and often tied to a specific embodiment. A robot arm, a humanoid, a drone, and a self-driving car experience the world differently. Learning across those differences isn't trivial. The third challenge is compute. Video, 3D data, simulation, and robotics trajectories are all heavy. Training on them at frontier scale will be expensive. The fourth challenge is reliability. A demo can show that a system has learned impressive regularities. A product has to work across edge cases. This is also becoming an evaluation problem. A recent benchmark paper called WorldSimBench argues that visual realism is not enough. A video model can generate something that looks plausible and still fails as a simulator. The more important test is whether the generated future is consistent enough to support action-level decisions — in driving, manipulation, and embodied environments. That's a useful warning for investors. The benchmark isn't whether the video looks good. It's whether the model helps an agent act correctly. That gap is especially large in robotics and industrial settings, where customers care less about elegance and more about uptime, safety, cost, and integration. The fifth challenge is value capture. Even if world models become important, it's not obvious who earns the margin. It could be the model lab or the robotics company. It could be Nvidia or another infrastructure provider. It could be a cloud platform. It could be the company with proprietary deployment data. Or it could be the application layer, where the model is only one component in a larger system. That's the investment tension. The thesis may be right, while the business model remains unclear. IN CLOSING The world-model debate is really a debate about what AI is still missing. Large language models gave AI a powerful interface to knowledge and software. But acting in the world requires something more: prediction, memory, spatial understanding, and an ability to reason about consequences. That doesn't mean today’s world-model startups have already solved it. It doesn't even mean the winning approach will come from a standalone world-model company. The capability may emerge from multimodal foundation models. It may come through robotics. It may come from reinforcement learning, simulation, video, or some combination of all of them. But the direction of travel is clear. AI is moving beyond text. The next major frontier isn't just whether a model can answer a question. It's whether it can understand enough about the world to make a useful decision before it acts. That's why the money is moving. And it's why world models have become one of the most important bets in AI. Thank you for listening to this bonus episode of the UpNext AI podcast. We hope you enjoyed the content and learned something new. Before we wrap up, a quick note: this podcast is generated with the assistance of AI and is intended for informational purposes only. All referenced articles, research, and commentary remain the property of their original authors and publishers. If you found this episode useful, share it with a friend or colleague, and consider leaving us a rating or review. And don’t forget to check out our daily briefing episodes, where we distill the latest AI news and research into 10 minutes or less. Thanks for listening to UpNext AI — we’ll see you next time, when we cover what’s up next!