Latest / Embodied AI 101 / Robotic World Model: Neural Dynamics for Locomotion
Transcript
- Robotic World Model: Neural Dynamics for Locomotion. Legged locomotion is notoriously challenging for robots. Quadruped and humanoid robots have many degrees of freedom and unstable dynamics, making it hard to find controllers that work robustly on real hardware. Classic model-based controllers or model predictive control can work in narrow conditions, but tend to be brittle and require extensive hand-tuning. Model-free deep RL can learn dynamic gaits, but it often needs massive training data in simulation and still struggles with the sim-to-real gap. To address this, researchers are leaning on learned world models: neural networks that act as fast simulators of the robot’s physics. By learning the robot’s dynamics in a neural model, one can train controllers “inside” that model via imagined rollouts, reducing reliance on the true simulator or hardware. Recent work by Chenhao Li, Andreas Krause, and Marco Hutter at ETH Zurich exemplifies this approach. In their new paper, “Robotic World Model: Neural Dynamics for Quadruped and Humanoid Locomotion” (arXiv 2506.19827), the authors build a neural dynamics model of a legged robot and use it to train locomotion policies entirely in simulation. Critically, they show that policies trained in these imagined trajectories generalize better and transfer to real hardware with less performance loss than conventional baselines. Their contributions build on an earlier ETH project (“Robotic World Model: A Neural Network Simulator for Robust Policy Optimization” – NeurIPS 2025 workshop), but focus specifically on locomotion. The key idea is to learn an autoregressive dynamics model of the robot (a “neural dynamics” world model) and then plug that in as the environment for any model-free RL algorithm. Essentially, the robot’s state and action are fed into the neural model, which predicts the next state, then feeds that predicted state back in for the next step, and so on. This loop lets the model simulate long trajectories without running the real physics engine. By using self-supervised training on logged trajectories from the robot’s sensors (positions, velocities, etc.), the model learns to predict even complex, partially observable dynamics (such as foot contacts, joint limits, actuator delays, etc.) over many timesteps. In practice, this lets the researchers roll out hundreds or thousands of parallel hypothetical episodes in the world model, using them to train a policy (for example with an algorithm like PPO) as if it were in the real simulator. The policy learns to maximize reward in the model’s imagination, then is deployed on the actual robot. This “model-based” technique has several advantages. First, the world model itself is a fast simulator. As the DayDreamer team observes, a learned model “can be thought of as a fast simulator of the environment”. Instead of running a costly physics engine, the policy can generate millions of transitions per second by evaluating neural nets on the GPU. This allows massive parallelization and large-batch RL training. Second, because the model is learned from data, it can capture complex, stochastic effects in the real robot’s dynamics. Third, decoupling the model from the policy lets you use standard model-free RL algorithms with minimal modification. The policy just “sees” the model as the environment: at each step it outputs an action, gets a predicted next state and reward from the model, and continues. In effect, this uses the model to generate imagined rollouts for RL. As previous work (e.g. the Dreamer family of algorithms) has shown, “Dreamer learns a world model from a replay buffer of past experiences, [then] uses an actor-critic algorithm to learn behaviors from trajectories predicted by the learned model”. The ETH team’s approach is conceptually similar, but they apply it to the high-dimensional locomotion domain and show it can overcome the usual long-horizon errors. Concretely, the paper describes the architecture of the Neural Dynamics World Model. It consists of neural network modules that take in the robot’s kinematic state (joint angles, velocities, base orientation, etc., as sensed by IMUs or encoders) and the control action, and then predict the next state. The model is autoregressive, meaning it uses its own previous outputs as inputs for multi-step prediction. During training, the model sees sequences of real robot states and actions and learns to minimize prediction error. Crucially, the model is trained self-supervised on sequential data. With this training regime the model learns to “imagine” the robot’s behavior over many timesteps. By including noise and regularization, the model becomes robust to small errors; the result is a long-horizon predictor that does not drift away immediately. Indeed, the authors highlight that their model achieves “reliable long-horizon predictions without relying on domain-specific inductive biases”, handling complex partial observability and noise. Once the neural dynamics model is trained, the next step is policy learning via imagination. Instead of deploying the policy on the real robot or even a physics simulator, they roll out trajectories entirely within the learned model. In each imagined episode, the policy proposes actions, the world model generates the next state, and the policy observes that to act again. The reward is computed as if in the real task (forward velocity, stability, energy use, etc.). The policy is then updated by any model-free RL algorithm – essentially performing RL inside the simulated world the neural network provides. This is analogous to how Dreamer and other “world model” methods have been used: behaviors are learned from rollouts imagined in the model’s latent space. By doing so, learning becomes much more data-efficient: each real-world trajectory is “amplified” into hundreds of simulated ones. The authors integrate this with NVIDIA’s Isaac Lab simulation framework (built on Isaac Sim). Isaac Lab is an open‐source, GPU-accelerated simulation environment for training robot policies at scale. By running the learned world model on the same GPU platform, they can train agents orders of magnitude faster than real time, parallelizing many agents in the model. This setup lets the policy rapidly improve in the learned model before any trials on the physical robot. A key claim is that combining a learned dynamics model with model-free RL leads to better sim-to-real transfer than some standard model-based baselines. In the experiments, the ETH team tests both quadruped and humanoid locomotion tasks in Isaac Lab. (The quadruped is likely similar to the ANYmal platform used in their prior work, and the humanoid is a complex 3D biped.) They compare against baselines that either use a naive model-based RL or standard PPO in the physics simulator. The neural-dynamics approach outperforms these baselines when ultimately deployed on hardware. In other words, policies trained via the imagined rollouts of the world model show higher reward and robustness on the real robot, with notably less loss between sim and real performance. This matches the intuition that a learned model, while approximate, can incorporate realistic noise and variability, making the policy less overfit to an ideal simulator. In their earlier work on ANYmal manipulation, the authors emphasize that robust world models help bridge the reality gap by addressing compounding errors and noise. Behind the scenes, the success depends on the world model’s quality. The model must capture subtle dynamic effects (like slip or torque limits) well enough to train a safe gait. The ETH team uses techniques like a “dual autoregressive mechanism” (as in their previous RWM paper) to enhance accuracy over longer horizons. Practically, they likely hold out sequences of simulated data and ensure the model’s predictions remain well-calibrated many steps ahead. If the world model were poor, the imagined rollouts would diverge and RL would optimize for incorrect behaviors. Instead, by robust training the model, they effectively create a fast, reusable simulator. As one of their sources notes, the world model acts as a “fast simulator” that the robot learns autonomously, starting from scratch and improving over time. The policy training then becomes parallelizable at scale. This approach fits into a recent trend of “world model” methods for robotics. Earlier work like DayDreamer has shown that model-based RL can crank up data-efficiency on real robots. DayDreamer even trained a quadruped to right itself and walk within an hour of real-world time. The ETH team’s contribution is a specialized design and experimental demonstration for locomotion in Isaac Lab. By pairing neural dynamics with large-scale RL, they can handle much longer and more complex tasks than naive algorithms. In summary, the “Neural Dynamics World Model” is a learned simulator that supports locomotion learning. It uses neural nets to predict multi-step robot states, enabling long-horizon autoregressive prediction. Policies are trained within the model’s imagination, allowing efficient data reuse. Experiments show this yields policies that transfer to both quadruped and humanoid hardware with superior performance compared to standard baselines. In effect, the world model reduces the sim-to-real gap by capturing realistic dynamics in a training-friendly form. This work demonstrates how learned world models – combined with model-free RL – can scale to complicated robotic systems. It paves the way for adaptable, efficient locomotion controllers that learn in silico yet work in vivo, fulfilling the promise that a solid world model is “crucial for enabling efficient and scalable robotic control in real-world environments”. References: This work builds on the ETH Zurich “Robotic World Model” framework, uses NVIDIA’s Isaac Lab for simulation, and follows the Dreamer-style training paradigm where policies learn from predicted trajectories. Prior studies like DayDreamer have also treated the learned model as a “fast simulator” for robot learning. The reported results highlight advances in data-efficient legged locomotion via learned dynamics models.