Latest / Embodied AI 101 / SuperMimic: Training Humanoid Motion Control on One GPU
Transcript
- SuperMimic: Training Humanoid Motion Control on One GPU. SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model • Qiayuan Liao et al. • UC Berkeley • CoRL (oral) • 2026. Cold Open. Imagine you have a new humanoid motion library and one afternoon to find out whether your controller can learn it. You want walking, jumps, floor contacts, and acrobatics in the same policy. Do you reserve a GPU—or a cluster?. That is the provocation behind SuperMimic. The Berkeley team reports surpassing NVIDIA’s released SONIC tracker after three hours on one H100. SONIC’s reported training run used 128 L40 GPUs for seven days. The surprising move is to add another learning problem: train a model of the robot’s dynamics alongside the controller. At first glance, that sounds like extra work. You already have a simulator. Why learn another representation of its physics?. The answer turns on what the controller receives from that representation: predicted experience, plus a direction for improving its actions. Which of those benefits makes the difference—and what exactly does the headline comparison measure?. The Idea. The reusable object is the body’s response to actions. A different reference motion changes the desired trajectory; it does not replace the robot’s joints, masses, or contact geometry. A learned dynamics model makes that shared structure explicit. Ubisoft’s SuperTrack established this recipe for simulated characters: learn a differentiable approximation of motion, then train a controller through it. The simulator supplies examples; the neural surrogate supplies derivatives. Consider a landing. A sampled rollout tells an optimizer how well a particular attempt scored. Differentiating a predicted rollout also estimates how changing an earlier motor command would change the later trajectory and tracking objective. That gives credit assignment a directional signal: adjust this action because of its predicted effect on that outcome. The trajectory becomes something the optimizer can differentiate, rather than simply observe and score. Sharing across motions already exists in generalist policy weights. SONIC trains a unified controller and connects it to teleoperation and vision-language-action systems. SuperMimic makes the shared dynamics an explicit component of the learning process. Its loop alternates simulator collection, world-model fitting, and policy learning. MuJoCo Warp generates transitions; a state-predicting multilayer perceptron learns from replay; imagined trajectories train the actor. The actor encodes upcoming reference frames, combines them with recent proprioception, and outputs joint targets. The world model keeps learning throughout training and is held fixed during each actor update. Gradients pass through that learned model, not through MuJoCo. The dynamics model also receives randomized physical context, including friction and calibration offsets. The actor practices with noisy sensor observations. This separates information useful for learning the dynamics from information available to the deployed controller. Keep the two kinds of training data separate. BONES-SEED supplies motion-capture trajectories already retargeted to the Unitree G1. Those specify desired behavior. Simulator transitions describe the consequences of commands. A reference says where the body should go; a dynamics example says where it went after an action. Those are different forms of supervision, serving different jobs. A critic supplies the value beyond the short imagination window. This follows the short-horizon actor–critic principle in SHAC: differentiate a manageable stretch of dynamics, then use a learned value estimate to account for what comes afterward. Think about the landing again. The useful endpoint is a state from which the robot can keep moving, not merely a good-looking pose at the final predicted instant. The critic brings that future into the optimization. The lineage also includes SuperDyno, which scaled differentiable neural dynamics to tens of thousands of motion clips. SuperMimic advances that line through a jointly trained system that reaches physical humanoid deployment. Its contribution is the combination of shared dynamics, imagined experience, direct policy gradients, and deployable control—not the invention of learning through a neural simulator. The Result. The number to remember is three hours. That is the reported training-loop budget to exceed SONIC’s released-checkpoint return on the filtered BONES-SEED training library: 129,514 motions. Simulator collection, world-model fitting, and policy updates are included. Checkpoint evaluation happens afterward. The benchmark runs each motion from its first frame in deterministic simulation, with observation noise and domain randomization switched off. Return is the accumulated reward normalized against the episode’s maximum possible reward. A failed motion is one that triggers the training termination rule. This measures coverage of the training library. The underlying dataset contains everyday movements, dance, combat, and stylistic variations, including original and mirrored performances. That diversity makes the shared-dynamics question concrete: many different desired trajectories must be executed by the same articulated body. The ablations separate the proposed mechanisms. On LAFAN1, a controlled PPO-in-simulation baseline shares the observations, reward, network sizes, and evaluation protocol. SuperMimic reaches better tracking performance sooner. A second comparison runs PPO inside the learned world model; differentiating through that model reaches PPO’s best training return with substantially fewer imagined steps. That second comparison is especially informative. If the benefit came entirely from replacing an expensive simulator with a faster predictor, PPO operating inside the predictor would capture the essential advantage. The remaining difference points toward the policy update itself. The model is valuable both as a source of experience and as a source of derivatives. That interpretation follows directly from how the ablation separates rollout generation from optimization. The dataset-size sweep provides another positive signal: prediction error remains approximately steady as the motion library expands, while compute to a fixed return grows slowly. For practitioners, that is the scaling behavior worth studying. The interesting question becomes which additional motions contribute useful information about the body, and which mostly repeat dynamics the model already understands. I would use that question to organize the next set of data-selection experiments. Finally, the authors deploy one general tracker on a physical Unitree G1, demonstrating a skipping jump, a side roll, and flips without per-motion tuning. That puts the learning architecture in contact with the engineering problem that matters: producing commands that move an actual humanoid through demanding maneuvers. The Biggest Weakness. The headline compute comparison is not a matched time-to-quality experiment. It compares SuperMimic’s time to a target score with SONIC’s entire reported training run. That does not establish how much compute an equivalently configured PPO system would need to reach the same score. SONIC trains on a larger motion collection and learns a universal representation for robot, human, and hybrid control inputs. Its optimization combines PPO with auxiliary representation losses. The comparison also crosses GPU types. These are different training systems evaluated on a shared behavior benchmark, rather than two optimizers substituted into an otherwise identical pipeline. The missing counterfactual is straightforward: what would a well-tuned model-free tracker cost on this exact library and objective? Perhaps considerably less than SONIC’s completed cluster run. Perhaps still vastly more than SuperMimic. The published comparison cannot assign that number. This matters because a reported training budget and a measured requirement are different things. A system’s total expenditure tells you what its developers spent to produce a particular artifact. A controlled learning curve tells you what reaching a specified quality level costs under specified conditions. The latter is what you need to turn this comparison into a portable algorithmic speedup. The controlled LAFAN1 experiment supplies meaningful evidence in SuperMimic’s favor. It strengthens the case for the method, while leaving the magnitude of the large-library advantage unresolved. The decisive experiment would hold the motion set, observations, rewards, controller capacity, simulator, and hardware constant; give both approaches comparable tuning effort; and measure total training cost against tracking quality on the large library. Until then, the demonstrated single-GPU training budget is the firm result. The advertised GPU-hour multiplier is a comparison between complete systems, not an isolated measurement of the algorithm. Verdict. Humanoid motion-tracking researchers should pay attention. SuperMimic makes explicit dynamics learning a serious candidate for the training stack. The architectural attraction is that one learned object serves two purposes: predicting experience and supplying sensitivities for policy optimization. That is a productive continuation of the SuperTrack and SuperDyno line of work. For a smaller robotics group, the practical prize is a faster hypothesis loop. Treat a broad controller as something you can repeatedly rebuild while testing ideas. Compare motion-selection strategies, target representations, and training objectives. Ask whether more examples of ordinary locomotion buy as much useful behavior as rarer transitions involving different contact patterns. Those questions become particularly interesting with a structured motion resource. BONES-SEED distributes G1 trajectories alongside motion categories and annotations, making dataset composition an explicit experimental variable. For world-model researchers, the compelling question is what makes a dynamics model useful as a teacher. Prediction accuracy is one diagnostic. Another is whether its sensitivities lead the actor toward better control. I would spend the additional iteration budget studying that relationship: when does a better trajectory prediction produce a better policy update, and when does changing the optimization horizon matter more?. SHAC’s combination of a truncated differentiable rollout and a learned critic provides a useful conceptual starting point for that investigation. It treats the prediction window and the value of its endpoint as complementary parts of the learning algorithm. There is also a concrete community entry point. The independent SuperMimic-Reproduced project provides PyTorch code, a downloadable checkpoint, evaluation scripts, and experiment logs. Practitioners can inspect an implementation of the loop and use it to formulate their own experiments. The takeaway is architectural: share one dynamics model across motions, then use it for both fast experience and informative policy gradients. That is the idea to carry into the next humanoid-control project. Outro. The next result to watch is what happens when this recipe trains a wider family of control objectives, turning shared dynamics into a reusable asset across tasks as well as motion clips. For now, hold onto the engineering question SuperMimic raises: are you paying for more trials when you could be extracting more useful information from each trial?. A humanoid can be asked to perform thousands of different movements. Its goals change constantly. The physical relationship between its actions and its motion is something worth learning—and sharing. This is Embodied AI 101.