Latest / Embodied AI 101 / GreenWAM: Can One World-Action Model Learn Across Robot Bodies?
Transcript
- GreenWAM: Can One World-Action Model Learn Across Robot Bodies?. GreenWAM: A World Foundation Model for Robotics • Gennady Sidorov et al. • Brain Team, Sber Robotics Center • arXiv technical report • October 13, 2026. Cold Open. Imagine moving a bottle-packing skill from a humanoid to a tabletop bimanual robot. The goal stays the same. The arms, joints, cameras, and controller all change. What should the learning system recognize as shared?. A bottle lifting off a table is a visual event. An elbow flexing is a body movement. For a joint world-and-action model, these are two descriptions of the same interaction. GreenWAM connects those descriptions through shared anatomical coordinates and joint training on robot motion and its visual consequences. Its robotics training corpus spans twenty robot families. That sets up a more interesting question than whether a video model can produce a convincing robot movie. What happens when the model keeps practicing prediction while learning control? Does asking it to explain motion make it better at choosing motion?. Today on Embodied AI 101, we follow that question from the action interface to the ablation that gives this paper its strongest practical message. The Idea. NVIDIA’s Cosmos 3 Edge supplies the starting point. Its mixture-of-transformers architecture couples a reasoner, which handles contextual information, with a generator that produces continuous outputs such as video and actions. Shared attention connects them. Cosmos already supports forward dynamics, inverse dynamics, and joint video-action generation. GreenWAM specializes this foundation for robot bodies. During robotics training, the reasoner and video autoencoder stay frozen. The generator and action interface learn from roughly seventeen hundred hours of recordings, mostly real, spanning bimanual platforms and humanoids. Training examples align complete subtasks with camera streams, measured states, and language. The first important choice is the action interface. Robot-specific codecs place native coordinates into 79 anatomical slots. Corresponding elbows and finger joints occupy corresponding positions; unavailable coordinates are masked. All robots then share the learned action projections. Think about the distinction between matching tensor dimensions and matching meaning. Two vectors can have identical length while assigning completely different motions to the same index. An anatomical vocabulary makes the intended correspondence explicit before optimization begins. There is useful precedent in RDT-1B. Its physically interpretable action space organizes heterogeneous robot signals by their physical meaning, providing a consistent interface for multi-robot learning. The broader engineering principle is to spend some design effort deciding what should be shared, then let the network learn behavior through that structure. GreenWAM applies that principle to a model of both motion and the world. The second choice is the training question. A joint model can consume an observed scene and an action trajectory, then predict the future scene. Reverse the conditioning and it can infer movements from video. Supply the current scene and instruction, and it can generate the movement and visual future together. Cosmos supports these different directions of inference. Unified World Models, or UWM, made this relationship particularly clear: changing the noise levels assigned to video and action lets one transformer represent several conditional models. Forward prediction, inverse prediction, and policy learning become connected uses of the same experience. That is the intellectual lineage of GreenWAM’s seven-mode training mixture. This matters for supervision. A demonstration can answer “What did the robot do?” and “What changed as it moved?” In UWM, action-free video can also teach visual evolution. The learning objective determines which parts of the available evidence become useful for the controller. The final choice arrives during policy adaptation. GreenWAM copies the generator into an action expert. The action and video branches then specialize separately while communicating through attention; copying preserves their initial predictions. At runtime, the policy generates future video and state trajectories together. It executes part of the decoded trajectory, takes another observation, and replans. Actions therefore have both a command representation and a predicted visual consequence—a concrete design for keeping control connected to a learned model of the scene. The Result. The number to remember is eight percentage points: the gain in randomized RoboTwin success from retaining extra world-model modes during policy adaptation. Both variants already generate video with actions. They start from the same checkpoint and receive equal continuation-training budgets. The change is to retain forward-dynamics and image-to-video examples alongside joint prediction. The reported bootstrap interval for the randomized-scene gain excludes zero. That is an unusually actionable result. The intervention changes what the model must learn from its demonstrations. It does more than attach a video decoder to an action policy. One mechanistic hypothesis is that jointly generating an action and its outcome lets the two predictions accommodate each other, while predicting an outcome for a supplied action constrains that relationship differently. Complementary conditioning may help preserve representations that remain useful when the scene changes. This interpretation follows the distinction between joint and conditional learning emphasized by UWM. For anyone fine-tuning a generative controller, that suggests a concrete experiment: preserve complementary prediction tasks and measure their effect on closed-loop success. The headline benchmark figure is 86.8 percent success on LIBERO-plus, highest among the methods in the paper’s comparison. LIBERO-plus perturbs simulated manipulation tasks through viewpoint, layout, initial robot state, language, lighting, background, and image noise. Its design lets researchers change the conditions around a skill while preserving a recognizable task. This is why robustness benchmarks add useful information to standard task completion. They ask the controller to keep accomplishing the goal when familiar cues move. The question is behavioral: does the robot finish the manipulation under the changed conditions?. The world-model side is evaluated on WorldArena’s action-conditioned prediction track, where GreenWAM leads its comparison on trajectory, depth, and 3D accuracy. WorldArena measures multiple properties of generated interaction: motion, scene consistency, physical interaction, geometry, and control following. That evaluation complements policy success. One asks whether supplied motion produces the expected visual evolution; the other asks whether executed behavior achieves the instruction. Testing both gives a joint model two meaningful responsibilities. On the physical Green humanoid, GreenWAM beats the authors’ adapted π0.5 and Cosmos baselines on packing-instruction success. They also test a distilled, single-step policy on the robot and time inference on an RTX 4090. The Biggest Weakness. The cross-embodiment benefit is not isolated from target-robot adaptation. That is the biggest gap between the results and the broadest reading of the foundation-model claim. Twenty robot families describes the training corpus, not twenty unfamiliar bodies controlled by an unchanged policy. The benchmark results come from target-specific adaptation, and the released weights are separate benchmark checkpoints. The Franka example sharpens the issue. A single-arm Franka is absent from GreenWAM’s robotics-stage corpus. Yet Cosmos 3’s own action-training data include Franka Panda recordings. A body missing from the added dataset is not necessarily missing from the inherited model’s experience. Likewise, “zero-shot” on LIBERO-plus refers to evaluating a LIBERO-trained policy under task perturbations. It does not mean controlling a new robot without target-robot training. Why care? Because the distinctive representation claim is that corresponding anatomy makes experience reusable across bodies. Good visual priors followed by effective target-specific imitation learning could also yield strong benchmark performance. The reported comparisons do not cleanly separate those explanations. That is an attribution problem, not a complaint that adaptation is somehow illegitimate. The decisive experiment would hold out a robot family from all action pretraining, including the parent model, and measure how performance grows as target demonstrations are added. Compare anatomical sharing against embodiment-specific interfaces with matched training data and budgets. Keep the task distribution controlled enough that changing hardware is the main experimental change. Then ask whether the shared coordinates improve the learning curve: better initial performance, faster adaptation, or fewer demonstrations to reach a useful success rate. That would directly test the promise a robotics team cares about. Does experience from yesterday’s robots reduce the work required to train tomorrow’s robot?. Verdict. GreenWAM is worth studying if you build manipulation policies from video models, curate experience from multiple robot platforms, or work on evaluation for generative controllers. For policy researchers, the most productive takeaway is a training question: which predictive abilities should remain active while control specializes? Treat forward prediction as a capability you deliberately maintain. Design experiments around whether that maintenance improves behavior under changing scenes. This continues UWM’s view that policy and dynamics learning can share one statistical model. For dataset and systems engineers, the anatomical interface offers an explicit vocabulary for reviewing mappings. You can discuss what a coordinate means, which correspondences are intended, and how a new recording format enters the model. RDT-1B and GreenWAM both make physical meaning part of the interface design. The value of such a choice is practical clarity. When a new platform arrives, the team has a concrete mapping to inspect rather than treating a padded action vector as self-explanatory. That gives model researchers and hardware engineers a common object to discuss. The public release makes evaluation tangible. It includes weights for RoboTwin, LIBERO, CALVIN, and WorldArena, plus policy servers, benchmark clients, and rollout and scoring tools. Configurations and action statistics accompany the checkpoints. For an evaluation team, that creates a starting point for investigating the model’s behavior. The interesting unit of analysis becomes a matched pair: the world the controller predicts and the world its actions actually produce. You can ask which interventions improve agreement and which also improve task completion. WorldArena’s separation of predictive quality and functional utility provides a useful organizing framework for that investigation. Take this into your next experiment: build the learning problem so visual prediction and control continue to inform each other. Keep teaching the model what actions do while teaching it which actions to take. Outro. The next thing to watch is how teams use predicted futures during model development: as outputs to inspect alongside executed behavior, and as capabilities to preserve during adaptation. A robot policy gives you an answer to “What should move next?” A world-action model also gives you a visual expectation for that movement. Putting those answers next to each other creates a useful tool for experiments. GreenWAM’s strongest lesson is about that connection. Learning to act and learning what acting changes can remain parts of the same training problem. This has been Embodied AI 101.