Latest / Embodied AI 101 / SimToolReal: A Universal Dexterous Tool-Use Policy
Transcript
- SimToolReal: A Universal Dexterous Tool-Use Policy. Imagine a single robot policy that can pick up any handheld object and immediately use it as a tool – hammering, writing, sweeping, screwdriver-turning, flipping spatulas, or erasing – all without ever having trained on those specific tools or tasks. This is the promise of SimToolReal, a recent object-centric reinforcement learning approach from Kedia et al. (Stanford) that achieves truly zero-shot dexterous tool manipulation in the real world. In their approach, a single “universal” policy is trained only in simulation on a procedurally-generated menagerie of abstract tool shapes, under the always the same basic objective of reaching arbitrary goal poses. At test time, this policy can pick up unseen real tools and follow goal trajectories derived from a human video demonstration – all without any per-tool engineering or fine-tuning. In extensive benchmarks, SimToolReal outperforms naive baselines and matches specialist policies, while generalizing across 6 categories of common tools in over 120 physical test trials. In the entertainingly ambitious project page title “general dexterous tool manipulation,” the authors make good on their claim. They note that manipulating tools is a hard class of dexterity – objects are thin and awkward, require stable grasps and in-hand rotations, and often involve forceful contact. Traditional methods either rely on massive teleoperated datasets (hard to collect) or painstaking per-task engineering: you model the exact object geometry, design a specific reward function, and train a separate policy for each task. Instead, SimToolReal turns things on their head. It reduces every tool-use task to the same meta-problem: “move this object from its current pose to some goal pose.” In other words, the policy is goal-conditioned: at every step it sees the object’s current 6D pose and a desired goal 6D pose, and its task is simply to make progress. By randomizing the tool shape and goal pose in training, a single reinforcement learning policy learns to reorient many kinds of tools. This approach decouples the problem from the specifics of any single tool or task. In the authors’ words, SimToolReal uses “a universal goal of manipulating each object to random goal poses,” so that at test time it can perform “general dexterous tool manipulation … without any object or task-specific training”. Object-Centric Policy and Perception. A key innovation is the object-centric formulation of the observation space. Instead of giving the policy full images or the entire robot joint state, SimToolReal only gives it an abstract representation of the tool itself. Concretely, the policy’s input includes three minimal pieces of information: (1) the current 6D pose of the tool in hand, (2) a 3D bounding-box region specifying the likely graspable region on the tool, and (3) the current goal pose for the tool (relative to the robot frame). This deliberately strips away irrelevant details and focuses the policy on the core dexterity challenge. Moreover, the policy network is stateful (it has an LSTM core) so it can use the sequence of observations over time to infer hidden properties (like the object’s inertia or contact friction) and to plan multi-step adjustments. At deployment, extracting the 6D pose of an unfamiliar real tool is nontrivial. SimToolReal addresses this by using a modern vision pipeline: first the Segment-Anything-3D (SAM 3D) model segments a mesh of the object and estimates a coarse 3D shape, and then the FoundationPose tracking system obtains a full 6D pose of the tool from the RGB-D video. In effect, the vision system produces a mesh and pose for any novel object. This information (pose + grasp region) is fed to the policy. By doing this, the authors bypass the typical sim-to-real image gap: rather than trying to train a vision network for each tool, they rely on these pretrained “foundation models” to output the pose data, and train only the policy in sim. In their words, this gives “robust, scalable sim-to-real transfer across novel objects while bypassing visual domain gap challenges”. The action space is the robot control commands (likely joint torques or position deltas for an arm+hand, though the paper focuses on the high-level idea). Essentially, at each time step the policy outputs control signals that move the robot gripper and/or arm in order to align the tool with the goal pose. Because the goal pose itself moves over time (the policy is tracking a trajectory of goal poses from a demonstration), the policy must actively regrasp and reorient the tool as needed. Importantly, the goal poses come from a demonstration, not an analytic planning process: for a new task (say, writing a letter with a marker), a human performs the motion once while the RGB-D camera records the tool pose sequence. This sequence of goal poses is passed to the robot policy; the policy simply tries to follow it as best it can. In plex, SimToolReal “tracks goal trajectories extracted from RGB-D human videos” in zero-shot fashion. Training Environment and RL Algorithm. Training SimToolReal involves a highly parallelized simulation setup using Nvidia’s Isaac Gym. The authors generate thousands of procedurally random training environments on GPU, each with a different “tool primitive” and different randomized physics. Every training episode starts with the robot and a procedurally-generated tool on the table in some initial poses. They randomize a wide variety of parameters: the geometric proportions of the tool (lengths, diameters), its density and mass distribution, also the robot’s initial joint positions and object orientation, plus random delays and sensor noise in the loop. This extreme domain randomization means the policy never sees exactly the same setup twice, helping it learn robustness. For the objective, each environment selects a random goal pose for the tool and gives this to the agent as the target to reach. The reward is shaped to encourage smooth movements, strong stable grasps, and getting closer to the goal pose. Concretely, they use a “keypoint-based goal term” that penalizes the distance between a few fixed points on the tool and their corresponding locations in the goal pose, plus penalties for control smoothness and for fully closing on the object. Training such a universal policy requires scaling up reinforcement learning massively. The authors employ SAPG (Split-and-Aggregate Policy Gradients), a variant of PPO designed for huge parallel environments. SAPG was introduced at ICML 2024 as an on-policy algorithm that divides the many environments into batches (“chunks”) and reweights their gradients via importance sampling. The key point is that standard PPO (and other RL methods) tend to plateau when you have truly massive batches, whereas SAPG continues learning effectively. Indeed, Kedia et al. point out that without SAPG, training a policy at this scale failed – PPO simply saturated. With SAPG, they can aggregate experience from thousands of environments, improving exploration and convergence. They also use an asymmetric critic: during training, the value network (critic) is given privileged access to the full simulated state (including object velocities, exact mass, etc) even though the policy (actor) only sees the partial observation with object pose. This helps stabilize value estimates in the face of partial observability, accelerating training. (This is a common trick in sim-to-real: the critic sees the “ground truth” to learn faster, while the policy remains constrained to realistic inputs.). Putting it together, the training loop runs thousands of parallel IsaacGym environments, sampling all the randomizations, and optimizes the policy with SAPG PPO using the composite reward. The policy network (with LSTM) learns to control the robot to lift, rotate, and place the object according to the random goal. Over time it gets good at immediately grasping a new shape and orienting it to match an arbitrary pose. Crucially, the training does not include any of the real target objects or tasks. All knowledge must come from these synthetic primitives. Procedural Object Generation. A centerpiece of SimToolReal is the procedural generation of tool primitives in the simulator. Instead of hand-crafting models of each hammer, marker, eraser, etc., the authors programmatically create simple “tools” by random combinations of basic shapes. Specifically, each procedurally-generated tool has a “handle” (a cylinder) possibly plus a “head” (a cuboid or another cylinder), with random dimensions and connection positions. The head could be a flat rectangle (like a spatula blade) or a heavy block (like a mallet). They also randomize the material density of each part. This sampling scheme covers a huge variety of possible inertial properties. For instance, some shapes end up long and slender, others short and heavy, as needed to capture the wide spectrum of real tools. This random shape sampler is surprisingly powerful. By sweeping through lengths, diameters, and densities, they efficiently span the “diversity and inertial variations typical of real-world tools”. In effect, the policy never memorizes a single tool; it sees continuum of possible objects. The authors highlight that this obviates per-object asset modeling: you don’t need CAD models of every hammer and brush – you just make sure your random primitives cover those regimes. The LSTM policy then learns general skills like “pick up this flat long thing and rotate it,” rather than “here’s hammer A.”. The training tasks are all “reach a goal pose,” so the procedure also generates a random target pose for each new tool. In each episode, the policy must figure out how to grasp and rotate the unfamiliar shape to match that goal. This forces it to learn regrasping, wrist rotations, and even sliding grasps when needed (e.g. sliding a marker along its length to reach a goal orientation). Over many such trials, the policy builds skill at stabilizing and reorienting objects of odd shapes. DexToolBench: Evaluating Generalization. To rigorously test generalization, Kedia et al. introduce a new benchmark called DexToolBench. This benchmark comprises 24 “real-world” tasks across 6 tool categories (hammer, marker, eraser, brush, spatula, screwdriver) with 2 object instances per category (e.g. “claw hammer” and “mallet hammer”, two different markers, etc). Each task is derived from a human demonstration recorded with an RGB-D camera. For example, it includes tasks like swinging a hammer down, swinging it sideways, writing a smiley face with a marker, writing a letter “C” on a whiteboard, erasing a shape, sweeping a tabletop with a brush, flipping a pancake with a spatula, spinning screws with a screwdriver, and more. In total there are 12 distinct physical object instances and 24 scripted motion trajectories. Critically, none of these tasks or objects are seen during training. DexToolBench is zero-shot evaluation only. The policy never sees the specific tools in sim; it only saw its simple random cylinders and boxes. Yet at test time, we give it one of these real objects and the goal pose trajectory (extracted from the human video) and ask it to execute. The performance metric they report is “task progress”: the fraction of trajectory waypoints (pose goals) that the robot reaches within a tight error threshold (2 cm). Essentially, did the robot manage to follow the demonstration path accurately?. By design, DexToolBench covers a broad range of dexterity challenges: thin vs heavy tools, fine rotations vs simple motions, contact-heavy tasks, etc. For transparency and reproducibility, the authors have open-sourced all simulation assets, code, and even the full DexToolBench dataset (camera images, poses, masks, etc) on their GitHub. This will allow other researchers to benchmark zero-shot tool-use policies under the same conditions. Zero-Shot Real-World Results. On DexToolBench, SimToolReal achieves impressively high success rates, especially considering the zero-shot setting. In 120 real-world rollout attempts (2 tasks × 2 objects × 6 categories × 5 trials each), the policy “demonstrates high zero-shot task progress”. Tasks that require mainly translation (like sliding an eraser or pushing with a brush) are nearly perfect. For example, they report “near-perfect scores” on tasks like translating an eraser across a board or sweeping, since these only need a stable hold and little rotation. Even tasks involving in‐hand reorientation are often accomplished successfully. Of course, performance varies with tool geometry. The robot found thin or heavy tools more challenging. A flat spatula and a heavy hammer mallet, for instance, were harder for the vision and grasping system. The spatula blade is very flat and long, making reliable pose estimation and grasp grip tough; the heavy mallet loads the hand more strongly. The policy still tried – but the success fraction was lower on those. In their words: tasks requiring minimal rotation (like eraser slides) “yield near-perfect scores, while thin/heavy tools (e.g., flat spatula/hammer mallet) pose greater challenge due to grasp reliability and pose estimation limitations”. This makes sense: any painted policy has limits when the object is extreme. Overall, though, the average performance was quite high – far above zero and well beyond simple baselines (see below). The authors highlight that all of this was done without any fine-tuning or additional training on the physical system. The policy learned entirely in sim on synthetic primitives and was transferred directly. The vision pipeline (SAM3D + FoundationPose) handled novel real objects, and the LSTM policy handled the control. That it worked at all – let alone so robustly – is remarkable. It demonstrates that the object-centric abstraction and massive randomization indeed overcame the sim-to-real gap under these conditions. Baseline Comparisons. To put these numbers in context, SimToolReal was compared to two intuitive baselines:. Kinematic Retargeting. This baseline records the human demonstration as a full hand-object trajectory, and then attempts to follow it by inverse kinematics on the robot. Essentially it tries to place the robot hand at the same positions and orientations as the human did. Because it is purely kinematic, it does not attempt any dynamic interactions or grasp optimization. Not surprisingly, it fails early: it “cannot establish robust contacts and is incapable of grasping and interacting dynamically”. In practice, this means the robot might try to place its hand on the hammer but not actually close fingers to grasp, or it drops the tool during motion. Fixed-Grasp Trajectory Following. In this variant, the robot first uses an RL-trained strategy to establish a good grasp on the tool (one learned during training). Then it locks that grasp (treating the object as rigidly attached to the hand) and simply moves the arm through the recorded goal trajectory, using inverse kinematics on the arm while holding the hand shape fixed. This can only succeed if the task can be done without reorienting the object in hand. Of course, many of the DexToolBench tasks require in-hand rotation (like flipping a marker or turning a screw), so this method often fails. The authors note that “fixed grasp” is funneled: it works if you only need to translate the tool, but “fails to execute tasks involving significant object rotation”. Across every category of task that involved reorientation or contact, SimToolReal vastly outperformed both baselines. Figure 4 in the paper (excerpted in their page) clearly shows SimToolReal’s success fraction much higher on rotation tasks where the baselines drop to near zero. In quantitative terms, over all baseline comparisons across the benchmark, “SimToolReal outperforms prior retargeting and fixed-grasp methods by 37% in task progress”. In other words, if retargeting or fixed-grasp would achieve, say, 30% of waypoints on average, SimToolReal would reach around 41%. This is a substantial margin. The 37% figure comes directly from the authors’ reported numbers. In practice, this means that the learned policy consistently leverages its ability to regrasp and reorient. For example, on tasks like “flip the spatula blade 180°” or “twist the screwdriver”, the robot’s fingers could continually adjust grip and follow the video-goal sequence, whereas the fixed-grasp baseline simply couldn’t change the orientation. Likewise, on swinging a hammer, SimToolReal could pivot and even re-grip during the swing, unlike the kinematic baseline. Specialist Policies vs. a Generalist. The authors also trained “specialist” RL policies for comparison: each specialist is trained in simulation on a single known object and single task trajectory (i.e. no randomness besides some small domain randomization). These specialists perform very well on exactly their trained task – typically near 100%. But as expected, their performance collapses on anything new. A specialist hammer-swing policy trained only on a claw-hammer swinging down will fail if asked to swing a mallet-hammer or to swing sidewise, for example. SimToolReal, by contrast, is a generalist. Kedia et al. report that SimToolReal “achieves specialist-level performance on both training and novel objects/tasks”, without any adjustment. In their experiments, SimToolReal matched the specialists on the exact task and object that the specialists were trained on, and unlike the specialists it retained much of that performance on new objects and trajectories. Put another way, the specialists “degrade sharply if either object or trajectory changes,” whereas SimToolReal stays robust. This is a powerful result: the general-purpose policy comes very close to the best possible (specialist) on all the benchmarks, effectively collapsing the gap. The specialists are a useful lower bound on the sim-to-sim performance: SimToolReal trading only a small fraction of specialist proficiency on exact tasks, in exchange for enormous flexibility. Aligning Training with Zero-Shot Goals. The work also provides an interesting insight: the training objective of arbitrary pose-reaching is well-aligned with the downstream tool-use tasks. The authors measured a correlation between how well the policy improved on the simulated pose-reaching task and how well it did on the real DexToolBench tasks. They found “strongly correlated” improvements: as training reward went up, actual task success went up in tandem. This suggests that the meta-goal of reaching random poses indeed instills broadly useful manipulations (reorientation, grasp stability, etc.). In short, learning to “hit random goal poses” is an excellent proxy for learning general dexterity. Limitations and Future Directions. No approach is without limitations. SimToolReal currently assumes rigid tools and a fixed sequence of goal poses. That means it does not handle deformable or articulated tools, and it does not incorporate feedback to replan if the situation changes (e.g., an obstacle suddenly appears). Because the policy conditions only on the object pose and goal poses (and not on the rest of the environment), there is a risk of collision if clutter is around. The authors note explicitly that without environment information, “conditioning policies solely on object-pose can lead to collisions in cluttered or unmodeled scenes”. They also acknowledge that in tasks requiring very high force (like hammering nails), the current open-loop framework doesn’t guarantee full task completion – it is focused on precise pose control rather than force planning. Addressing these limitations is left to future work. Among the suggested directions are adding closed-loop replanning (so the robot can adjust goals mid-flight if needed), incorporating tactile or force feedback, and extending to non-rigid or multi-part tools. In more concrete terms: imagine allowing the policy to see not just the object but also a depth map of the scene, or to get touch readings from the fingers. That would help avoid collisions and adapt grasps if something goes wrong. Also, moving beyond rigid objects (e.g. manipulating a screwdriver with springs, or using a flexible tool) would require new representations beyond the static 6D pose. The authors discuss all these in their conclusion as next steps. Implications for Robotics. SimToolReal has important implications for the field of embodied AI and robotic manipulation. First, it provides convincing evidence that a single generalist policy can span what used to be considered many different tasks. The classical approach in dexterous robotics was “one controller per tool/task.” SimToolReal breaks that mold by showing “a single unified policy can achieve broad generalization across tools, tasks, and environments.” This suggests a future where robots are trained once in a rich simulator and can then handle new daily tools out-of-the-box, with no extra per-tool tuning. Second, the work highlights the power of object-centric conditioning. By abstracting the problem to “follow 6D poses of an object,” it bypasses many complexities (e.g. vision differences across domains). The success here validates the idea that object pose trajectories are a compact, task-relevant representation. In fact, Kedia et al. call this “object-centric position-based conditioning a robust abstraction for high-DoF manipulation”. It opens up possibilities to leverage more advanced goal specification methods: for instance, one could imagine conditioning the same policy on tool-use instructions from a language model that outputs target poses, or on trajectories derived from AR/VR demonstrations. Third, from a practical standpoint, SimToolReal greatly reduces engineering overhead. There is no need to design task-specific rewards or to model each object exactly. This lowers the barrier for deploying dexterous skills on new tools. As the authors note, this is “mitigating the need for costly object modeling, reward engineering, or teleoperated demonstration”. For a robotics lab or company wanting to scale up manipulation to many tools, that could be a game-changer. Finally, the approach fits well into recent trends: it relies on large-scale simulation (ISAAC Gym on GPUs), foundation models for vision (SAM 3D, FoundationPose), and powerful on-policy RL (SAPG) to bootstrap learning. It hints at a modular path: better vision front-ends or better RL algorithms could plug into this pipeline to further boost performance. Moreover, it opens the door to richer goals. The current goals are pose trajectories, but one could extend this to function-based goals (e.g. “move object so that its tip draws a circle in space”), or incorporate object affordance prediction. The SimToolReal framework is flexible enough to potentially adapt to those as well. Conclusion. SimToolReal represents a significant stride toward truly general dexterous manipulation. The key takeaway is simple yet powerful: abstract dexterous tool use as object pose-tracking and train one policy on a wide variety of synthetic shapes, and voilà – you get a robot that can pick up real tools and follow human demonstrations without ever having practiced them. In effect, Kedia et al. have taken several lines of research (sim-to-real RL, foundation vision for objects, procedural generation) and woven them together into a functioning whole. Their results are compelling: a single policy transfering zero-shot to dozens of novel tasks, surpassing naive baselines by a large margin and rivalling task-specific experts. It clearly shows that the paradigm of object-centric, goal-conditioned RL is viable for high-DoF manipulation. As they conclude, “SimToolReal provides compelling evidence for the efficacy of unified, object-centric RL policies in endowing robots with generalizable dexterous tool-manipulation capabilities”. In the world of embodied AI, this work pushes the boundary of what robots can learn in simulation and how flexibly they can apply it in reality. Looking ahead, we can imagine SimToolReal-style policies being components of larger systems. For instance, a household robot might observe a human using a tool, run FoundationPose on the video to get a trajectory of goals, and then hand that to SimToolReal to execute the task. Or combine it with natural language: “fetch a brush and sweep here” could translate to a sequence of brush poses it already knows how to follow. With further advances (touch sensing, re-planning, articulated objects), the approach could expand to entire tasks like setting a table with various utensils. In any event, SimToolReal has laid a solid foundation: it shows that one doesn’t need a myriad of policies for each new tool – a single cleverly trained policy can generalize across many. That is both scientifically intriguing and practically very promising for the future of dexterous robotics. References: Kedia et al., SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation (arXiv preprint, Feb 2026); Singla et al., SAPG: Split and Aggregate Policy Gradients (ICML 2024); (Other references as cited).