Latest / Embodied AI 101 / Play2Perfect: Dexterous Play Pretraining for Precise Assembly
Transcript
- Play2Perfect: Dexterous Play Pretraining for Precise Assembly. Imagine a robot hand learning how to build an IKEA chair by itself: before tackling the tight bolt insertions and careful alignments, it first learns to “play” with everyday objects. This is the idea behind Play2Perfect, a new two-stage reinforcement learning framework from Lum, Kedia, Liu, and Bohg (Stanford/Cornell 2026). In the first stage, the robot undergoes unsupervised skill play – learning general dexterous manipulation by randomly interacting with diverse objects. In the second stage, it is fine-tuned on specific high-precision assembly tasks. The result is dramatic: a policy that masters millimeter-scale insertions and even tabletop screwing tasks around 33× faster than learning from scratch. Moreover, this “play-then-assemble” training transfers straight from simulation to a real robot (zero-shot transfer), achieving about 60% success on extremely tight 0.5 mm-clearance insertions and over 50% on multi-part furniture assembly with screwing – all without further real-world learning. Figure: A multi-fingered humanoid robot extending its hand; dexterous multi-fingered manipulators promise human-like versatility for assembly tasks. Play2Perfect first lets such a hand “play” with objects in simulation to build basic skills (grasping, reorienting, reaching), then tunes the policy for the final precise steps needed in assembly. These results address a long-standing challenge in robotics. Human hands are amazing at fine assembly, but teaching a robot hand to do the same is notoriously difficult. Precise assembly tasks – think inserting a peg with a 0.5 mm fit or aligning tiny shelves – are contact-rich and sparse-reward by nature. It’s almost impossible to collect many human demonstrations (imagine showing the robot how to spin a screw or delicately place a cylinder); imitation is hard. Pure reinforcement learning gets stuck too: with only a “success/failure” reward at the end, the vast action space and tight tolerances make exploration infeasible. Prior work on robotic assembly has mostly dealt with these difficulties by heavy engineering: custom fixtures, simple parallel grippers, or specialized tools. In contrast, Play2Perfect takes inspiration from how a human child learns to build things – play. The key hypothesis is that a robot should first discover basic manipulation skills in a variety of contexts, and only then apply those skills to exacting assembly problems. In effect, Play2Perfect is an instance of pretraining for manipulation. The first phase is a task-agnostic dexterous play regimen: in a physics simulator, the robot hand is given many random objects and many random goals, and it practices moving and reorienting the objects. This goal-driven play uses reinforcement learning (a goal-conditioned policy) to maximize progress toward randomly sampled 6D object poses. By solving this generic reaching-and-reorienting objective over and over, the robot acquires reusable manipulation priors – stable grasps, controlled arm-hand coordination, in-hand rotations – that transfer to many tasks. In the second phase, those learned skills are refined on specific assembly tasks with sparse rewards. Practically, each assembly task is generated by “assembly-by-disassembly” from a CAD model: a sequence of goal poses that gradually insert or screw parts together. The policy, starting from the pretrained initialization, is then finetuned on reaching those precise goal sequences. The authors of Play2Perfect perform extensive experiments to show how and why this two-stage approach works. Strikingly, they report a 33× sample efficiency improvement over vanilla reinforcement learning from scratch, even when the scratch baseline is given a carefully engineered dense reward. In other words, learning all those low-level dexterous skills from scratch takes 33 times more data than if the robot first “plays” with objects to build up a repertoire. Qualitatively, the pretrained policies exhibit robust, finger-centric grasps and stable control, whereas scratch policies often resort to precarious, “balancing” maneuvers to fumble through the task (as noted by Lum et al., stories of trials relying on unstable tricks). Training details: In the play phase, the robot’s policy is conditioned on its own state, the current object’s pose, a target object pose, and a description of the object’s geometry. Each episode, a random object is spawned (the authors use simple shapes like cuboids and cylinders). A random goal pose for that object is sampled in 6-DoF (position + orientation). The robot then tries to move the object to the goal pose. The reward is shaped to encourage decreasing distance to the goal pose – for example, the paper describes a keypoint-based distance metric for full 6D pose. Crucially, the objects are randomized in size, weight, and center-of-mass so that the policy learns to handle many variations rather than overfitting to one shape. Over millions of simulated steps, the policy finesses grasps, rotates objects in-hand, and reaches target poses. By the end of pretraining, it has a general latent skill for robust in-hand pose control. In the assembly phase, the authors leverage the fact that most assembly tasks can be broken down into reaching a sequence of intermediate subgoals. Given a high-level assembly objective (say, “screw Part A into Part B”), they reverse-engineer a disassembly order via the CAD model and pose the task as reaching those steps in sequence. For example, to insert a cylinder into a hole, the first subgoal might be to align and insert nearly half way, and the final subgoal is full insertion. During fine-tuning, the robot gets a sparse reward upon reaching each subgoal (often only the final success is rewarded). Because the policy is already skilled at positioning objects from pretraining, the exploration problem is drastically simplified: instead of ball-parking random actions until the screw just happens to go in, the robot now searches around configurations that are close to a stable grasp and near the correct pose. Lum et al. ablate many factors in this approach. For pretraining, they find that object diversity really matters: exposing the policy to many shape and weight variations prevents it from latching onto shape-specific tricks. They also show that using full 6-DoF target objectives (position and orientation together) is better than simplifying to translation-only goals; the full objective forces the policy to coordinate arm and hand motions for complete reorientation. Another key is trajectory diversity: sampling completely random goal poses each episode (rather than following a fixed curriculum of movements) leads to faster learning. Finally, the goal precision during play matters: even though assembly fine-tuning will require millimeter-level accuracy, the play phase can initially allow a bit of slack (e.g. on the order of a centimeter) but not too much; training with tighter pose tolerances in the play phase helps prime the policy for the high precision needed later. Together, these design choices shape a pretraining regimen that primes the policy best for assembly. (We note that these detailed ablation results come from the technical appendix of the paper; the main abstract simply summarizes that these factors – diversity of object and goals, and precision of targets – are systematically evaluated.). The punchline is in performance. After pretraining, they run finetuning on a suite of challenging tasks, both in simulation and on a real robot hand. The comparison is stark: a network trained from scratch on the same assembly tasks falters without enormous data, whereas Play2Perfect quickly masters them. Quantitatively, training from scratch even with dense rewards took 33× more samples than fine-tuning the pre-trained policy. In practice, the pretrained policy often zeroes-in on the task after only a small amount of real-task interaction – slashing training time. Moreover, the policies learned by play-and-finetune are robust. For instance, a scratch-trained policy on a peg-in-hole task often only succeeded by precariously balancing the peg on a fingertip until it just squeezed in; even tiny disturbances would drop the peg. In contrast, the Play2Perfect policy had learned to grip the peg more solidly and guide it into the hole reliably. In the authors’ words, the pre-trained priors lead to “stable, finger-centric grasps capable of handling external force perturbations,” whereas scratch policies lean on brittle strategies. Importantly, Play2Perfect policies transfer straight to the real world. In the robot lab, the authors mount a depth-and-RGB camera to track the parts in 6D. They use a state-of-the-art 6-DoF object tracker (NVIDIA’s FoundationPose model) to get high-rate pose feedback (~60 Hz) of each object’s pose. With that, the robot can imitate the simulated policy in real time. Remarkably, without any additional real training, the policies achieve roughly the same success rates in hardware as in sim: about 60% success on a very tight insertion (0.5 mm fit clearance), and over half success on a multi-step furniture assembly task that involves inserting and screwing in parts. The sim-rel gap is small enough that they call it “zero-shot sim-to-real”. In the furniture task, the robot hand autonomously picks, aligns, and screws parts of a small wooden assembly – something that would normally stump any un-pretrained RL approach. These advances are built on and contrast with prior work in robotic learning. One common approach to sparse-reward tasks is to augment exploration with techniques like Hindsight Experience Replay (HER), which re-labels failed attempts as successes towards goals that were accidentally reached. HER and related tricks can help sparse tasks, but they still often struggle for extremely precise manipulations unless the robot already has a good grasp of basic skills. In contrast, Play2Perfect bakes in a skill-learning stage: it is as if the robot first practices without concern for the final task, and only later zeroes in. This is reminiscent of how we might first teach a robot to just hold and move objects (learning the hand-eye coordination) before asking it to perform fine assembly. The work also ties into trends of pretraining in robotics and RL. Just as large language models first train on generic text and then fine-tune on a specific task, here the robot first trains on generic play and then fine-tunes on the specific assembly task. Several recent papers have advocated unsupervised “play” or self-supervised learning for manipulation; Play2Perfect is a strong demonstration that such pretraining can birth substantial reuse. (For example, prior work on dexterous manipulation has looked at self-supervised skill generation from play, and here Lum et al. empirically dissect which aspects of that play are crucial when the end goal is precise assembly.). To implement Play2Perfect one needs the ability to simulate a rich hand-arm system and to generate goals. Lum et al. use a physics simulator (implementation details like exact hand model are in the full paper) with randomized objects. The play-phase policy is typically trained with a standard off-policy RL method (for instance, one could use SAC or DDPG with goal conditioning) on dense distance-based rewards for reaching each intermediate pose. The assembly phase then switches to sparse rewards: basically one big positive reward when the part has reached the final goal pose (and sometimes intermediate bonuses for stepping through the disassembly sequence). One nice trick is that the assembly plan is drawn automatically from the CAD model, so no human labeling of subgoals is needed. Finally, it is worth highlighting the scope of this method. Play2Perfect shines for tasks that require millimeter-level precision and sustained contact control: think industrial insertion, electronics assembly, or putting together small mechanical parts. It assumes we have a known goal configuration (a CAD-based target pose) and the ability to track objects in real time. It does not directly solve very long-horizon planning (beyond the multi-step assembly sequences it was given), nor does it handle unstructured tasks like cooking or cloth folding. But for what it does, the results are a big step forward. It shows that careful play pretraining can give dexterous hands the intuition they need to even attempt high-precision assembly. Looking forward, this suggests several avenues for roboticists. One could try scaling this idea by having the robot play with even more varied objects (maybe outside cubes/cylinders), or by incorporating vision (images instead of ground-truth poses) during play. Integrating tactile sensing in the play phase could let the robot learn touch-based manipulation. And on the application side, combining this approach with tool-use policies (e.g. pretraining includes using a screwdriver) could tackle even more complex build tasks. The Play2Perfect results are already a promising proof that dexterous robots can, in a sense, “play their way” to being construction workers. Key Takeaways: Play2Perfect demonstrates that pretraining a dexterous manipulator through unscripted play dramatically boosts the efficiency and robustness of learning precise assembly. By first learning general grasping and in-hand control on random objects, the robot acquires a manipulation prior that makes later fine-tuning on weakly rewarded tasks much easier. Design choices – like using fully 6D goals and diverse shape/randomized objects – turn out to make a big difference in that pretraining. The result is a policy that learns assembly 30+ times faster than learning from scratch, and that can even transfer directly to real hardware to perform tight insertions and screwing tasks. This work highlights the power of “learning to play” for embodied intelligence, showing that even millimeter-level robotics problems can be tamed by the right kind of self-supervised exploration and subsequent task-specific honing. Citations: Lum et al. 2026 (Play2Perfect). (Additional references: foundational RL methods like Hindsight Experience Replay for sparse goals, and modern 6D pose tracking like FoundationPose.).