Latest / Embodied AI 101 / MolmoBot: Can Simulation Alone Deliver Zero-Shot Manipulation?
Transcript
- MolmoBot: Can Simulation Alone Deliver Zero-Shot Manipulation?. MolmoB0T: Large-Scale Simulation Enables Zero-Shot Manipulation • Abhay Deshpande et al. • Allen Institute for AI and partner universities • CoRL • 2026. Cold Open. Imagine bringing a manipulation policy into a room it has never seen. You put a mug beside a bowl, choose a camera position, and ask the arm to move the mug. Before touching the robot, you face a familiar research decision: how much demonstration data should you collect in this room?. What if the answer were none?. That is the experiment behind MolmoBot. Its robot-action training comes from simulation, and its released DROID policy reaches the physical robot without training on real robot trajectories. The interesting question starts upstream of the neural network. Instead of organizing more teleoperation sessions, imagine improving a program that constructs scenes and generates demonstrations. Every change to that program changes what the policy gets to practice. What should it practice? Different objects? Different camera placements? Different approaches to the same grasp?. To understand MolmoBot’s real-world result, we need to follow its training experience backward—from the gripper to the machinery that produced the data. The Idea. Here, zero-shot describes transfer into the real world. The manipulation skills are learned from simulated demonstrations, while the perception-and-language foundation comes from a pretrained Molmo2 model. The training recipe combines those representations with synthetic action supervision. The released MolmoBot-Data contains approximately 1.7 million expert demonstrations. Its task configurations cover picking, placement, placement conditioned on attributes or spatial relations, and articulated-object manipulation across Franka and Rainbow Robotics platforms. The downloadable episodes contain actions, visual observations, and other sensor data. The generator, MolmoBot-Engine, operates inside MolmoSpaces. It constructs task instances across procedurally generated environments and uses task-and-motion planning to produce expert trajectories. Think of it as an automated demonstrator: arrange a task, generate a solution, and record what the robot observes while executing it. That division of labor is important. The generator solves the teaching problem; the learned policy solves the deployment problem. You can invest engineering effort in producing useful demonstrations, then reuse the resulting experience across policy architectures. The planner’s role is to supply training behavior, rather than to become the deployed robot’s entire intelligence. Randomization runs throughout the pipeline. Objects, placements, viewpoints, lighting, textures, and physical dynamics vary across training examples. The intended effect is to make successful behavior depend on task-relevant relationships rather than a particular visual arrangement. Camera variation is especially prominent in the released system. Picture the same grasp viewed from different sides of a table. The image coordinates change dramatically, while the physical objective remains the same. Training across those variations asks the policy to connect observations to actions through something more stable than a memorized camera view. That is the intuition behind the recipe. The main MolmoBot policy processes multiple frames of RGB observations and a task instruction through the Molmo2 backbone. A flow-matching action head combines that representation with robot state to predict action sequences. This follows the broader architectural direction established by Physical Intelligence’s π₀: connect pretrained visual-language understanding to a continuous-action generator. In plain language, flow matching learns how to transform a noisy candidate action sequence into a coherent movement. Predicting a sequence gives the controller a short coordinated plan. The robot executes part of that plan, observes again, and generates the next chunk. Perception and control therefore remain connected during execution. The release also contains MolmoBot-Pi0, which follows π₀’s architecture, and MolmoBot-SPOC, a compact transformer alternative. All three policy classes use behavior cloning on synthetic demonstrations. They provide different ways to consume the same underlying training resource. There is a clear research lineage here. GraspVLA previously demonstrated synthetic-action pretraining for real-world grasping, combining simulated trajectories with visual-language supervision. Its pipeline also used planning, domain randomization, and large-scale rendering. MolmoBot develops that simulation-first direction into an openly released system spanning tabletop and mobile manipulation. The central engineering idea is therefore a reusable source of robot experience. Once you can manufacture demonstrations, data composition becomes something you can program, inspect, and experimentally change. The Result. The headline comparison is 79.2% real-world pick-and-place success for MolmoBot, versus 39.2% for π₀.₅-DROID. These results come from the authors’ evaluation suite on DROID hardware: forty tasks and 120 trials per policy, distributed across kitchen, workroom, bedroom, and office settings. Multiple physical robots and institutions participated. That is a substantial separation on a concrete physical task. The policy must connect the instruction to the relevant object and destination, then turn that interpretation into a completed manipulation. Recognition, language grounding, approach geometry, grasping, transport, and release all contribute to the outcome. The baseline is also recognizable to practitioners. Physical Intelligence distributes π₀.₅-DROID through OpenPI as a checkpoint fine-tuned on the DROID dataset and intended for inference on that platform. This makes the comparison relevant to someone choosing an available policy for an existing Franka setup. There is an architectural control inside the study. MolmoBot-Pi0 outperforms π₀-DROID while retaining the same architecture. The ablations also find that more demonstrations improve picking, and that absolute joint targets transfer better than delta targets on the Franka, despite similar simulation performance. Together, those findings make the data recipe and control interface substantive experimental variables. A new backbone is one way to improve a robot. Changing the experience used to train it is another. Changing what its action outputs mean can matter at the physical interface. For a working roboticist, that last point is particularly useful. An action representation is part of the deployed system, not merely a convenient label format. It determines how a prediction becomes a command and how the next observation closes the loop. The ablation directs attention to that interface. The bigger scientific result is that generated trajectories can support a useful observation-to-action mapping on physical hardware. The policy has learned from virtual interaction, then used that learning to manipulate real objects under the study’s evaluation conditions. This changes the research workflow worth considering. Alongside asking which model to train, ask which experiences to generate. A data engine makes that question concrete: you can change the training distribution deliberately, retrain, and measure what those changes buy on the robot. The Biggest Weakness. The strongest objection is that the broad manipulation claim outruns the breadth of the real-world evidence. The quantitative mobile door-opening experiment contains nine trials across three pull doors. The door specialist succeeds twice, both on the same door. Several trials encounter hardware faults. Those faults complicate diagnosis of the learned policy, while leaving little evidence of reliable end-to-end mobile transfer. This matters because mobile articulation is central to the paper’s broader ambition. A convincing tabletop result and a convincing mobile-manipulation result require their own evidence. Success in one setting cannot establish reliability in the other simply because both policies consumed data from the same generator. There is supporting context in Ai2’s later MolmoAct2 paper, published in May 2026. On its different real-world DROID tasks, MolmoBot’s advantage over π₀.₅-DROID is much smaller. That evaluation includes partial credit for a multi-object task, so its aggregate is not directly comparable with MolmoBot’s original benchmark. It nevertheless reinforces the interpretation that the size of the advantage depends on the task distribution. The unresolved question is how far the demonstrated transfer extends across manipulation regimes. Visual variation, object variation, and variation in required physical behavior are distinct dimensions. A headline aggregate can establish success on the tested mixture without establishing coverage of all three. The decisive next experiment would freeze the policies and evaluate a predeclared collection of unfamiliar articulated objects and mobile manipulation tasks. Vary handle geometry, approach direction, and required base motion systematically. Record task completion and hardware interruptions separately. That would test the broader thesis directly: whether the same simulation-first recipe delivers dependable transfer across materially different physical interactions. Verdict. MolmoBot deserves attention from teams building language-conditioned manipulation policies, especially researchers who want control over their training data. For an existing DROID-style Franka setup, the practical entry point is straightforward: the released policy uses a wrist camera and one external camera, and the repository provides instructions for serving the model and running robot evaluations. The release also includes separate implementations for the policy families. Treat the checkpoint as an experimental baseline you can put alongside your current system. Ask how it behaves under your observation setup and task distribution. That is a much more productive starting point than debating simulation and real data in the abstract. The data engine is the longer-term research asset. MolmoSpaces exposes configurable generation and evaluation pipelines, with experiment configurations specifying scenes, robots, and tasks. MolmoBot’s released dataset provides the corresponding training episodes for its manipulation configurations. That combination enables useful counterfactual experiments. Keep the model fixed and change what it practices. Keep the training budget fixed and change the mixture of task situations. Study which generated experiences improve the behaviors you actually care about. For VLA researchers, this is an invitation to treat demonstration generation as part of model design. The teacher determines which approaches, transitions, and outcomes the student sees. Designing that teacher is a technical contribution with downstream consequences for the learned controller. For robotics infrastructure teams, the appeal is repeatability. An explicit generation configuration gives collaborators something concrete to inspect and modify. It turns discussions about data quality into experiments that can be rerun. The takeaway is direct: MolmoBot makes simulation a credible primary source of action supervision for real pick-and-place, and releases the machinery needed to investigate that approach. The useful artifact is both a capable policy and a way to keep asking better questions about what robots should practice. Outro. Watch what happens when these data engines start choosing the next training situations themselves. Imagine a loop that identifies a behavior worth improving, generates targeted experience, retrains the policy, and checks whether the physical result changes. The interesting optimization target becomes the usefulness of the next demonstration. That is the research direction MolmoBot makes tangible. We can ask more than how to fit a policy to the data already on disk. We can ask what experience would make the policy better—and build a system that produces it. For embodied AI, learning to design that experience could be as consequential as designing the next network.