Latest / Embodied AI 101 / DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation

Transcript

DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation · Embodied AI 101

• DexNDM: Closing the Reality Gap for Dextereous In-Hand Rotation. Manipulating objects in-hand – for example, turning a screw or reorienting a key – is a tantalizing yet unsolved problem in robotics. Recent works have shown impressive feats of dexterity (think spinning pens or lightbulbs in the air), but these successes have often been limited to simple shapes or lab-specific setups. The core issue is the reality gap: the complex, contact-rich dynamics of a multi-finger robot hand interacting with a free object creates huge discrepancies between simulation and the real world. Small differences in friction, delays, or unmodeled forces can make a policy that works in sim catastrophically fail on hardware, especially for challenging in-air rotations. In practice, prior approaches to in-hand rotation have typically targeted simple, large objects (aspiring to a few centimeters in size, low aspect ratio) or have relied on conservative motions. For example, the recent AnyRotate system rotates objects under gravity-invariant conditions, but it was demonstrated on fairly modest objects (at least 5 cm in size and modest length). More broadly, Chen et al.’s Visual Dexterity system (Science Robotics 2022) showed impressive reorientation of novel shapes, but again on objects with support (e.g. a table) and without dealing with very small or long thin items. In short, as Liu et al. (ICLR 2026) note, generalized in-hand rotation remains a significant challenge because “the complex, contact-rich dynamics of dexterous manipulation create a ‘reality gap’ that has limited prior work to constrained scenarios involving simple geometries, limited object sizes and aspect ratios, constrained wrist poses, or customized hands”. DexNDM – Dexterous Neural Dynamics Model – is a new sim-to-real framework that tackles this challenge head-on. In this work, Xueyi Liu and colleagues from Tsinghua University et al. present a single learned policy that, after simulation training, can rotate a vastly broader range of objects in the real world than ever before. It handles long slim objects (up to ~20 cm, aspect ratio >5), tiny items (around 3 cm), and bizarre complex shapes (like animal statues) – all in midair, under gravity, and from arbitrary wrist orientations. They achieve this by combining a careful sim-space training pipeline with a novel joint-wise neural dynamics model and a clever autonomous data collection strategy to close the gap. In experiments, DexNDM’s policy dramatically out-rotates prior methods: e.g., on a test cube object it spins about six times (≈39 radians) before dropping, compared to only one or two spins (≈6–8 radians) by AnyRotate; and it even matches or beats state-of-art Visual Dexterity on their benchmark objects. Importantly, DexNDM only uses a modest amount of biased real data (no carefully-labeled successes at all), yet it robustly succeeds on many hard cases that failed before. We’ll walk through what they did and why it matters. From Specialists to a Generalist Policy. DexNDM’s pipeline starts entirely in simulation. The goal is to learn a single “generalist” dexterous policy that can reorient many different objects along a desired axis under rich, multi-axis and multi-wrist conditions. To handle the difficulty, the authors break it into two stages (with details in Sec. 3.1–3.2 of the paper). First, they create a set of category-specific “oracle” policies by reinforcement learning. Each oracle is trained (in simulation) on a particular object category (e.g. rods, cuboids, cylinders, animals) with privileged observations (like full object state). The problem is formulated as a finite-horizon POMDP: at each timestep the policy sees a history of proprioceptive state (joint angles/velocities), fingertip and object positions, per-joint contact and force signals, and the commanded rotation axis. The action is a joint-position offset (a small change ∆a for each joint angle target), which is converted to torques by a PD controller. The reward is a weighted combination that encourages rotation about the desired axis, penalizes unwanted motions, and – crucially – includes an intermediate waypoint reward for very challenging cases. In practice, rotating a 20 cm rod purely by end-of-episode reward is too hard, so they add a term r_goal that guides the object toward a waypoint along the axis; this nudges the RL policy to make progress even on the toughest shapes. After training these oracles on each category (using plenty of simulation data and privileged info), the second stage distills them into one real-world deployable generalist policy. They do this via behavior cloning (BC): roll out all the oracle policies, collect only the successful trajectories (i.e. those that actually rotated the object substantially), and train a single network to imitate them. In other words, they throw out failed or partial attempts and simply learn from the best oracle demonstrations. Attempts to use DAgger-style interactive distillation failed (the task is too hard for naïve on-policy blending), so pure BC on successes was the recipe. This yields a single policy that, from purely hand-centric inputs (proprioceptive histories, contact signals, wrist pose, and target axis), can execute the complex finger gaits needed to spin objects in sim. So at this point, DexNDM has a generalist simulation policy π_base(⋅) that works on many shapes and axes in the sim. But it still suffers the inevitable reality gap once on hardware. Neural Sim-to-Real via Joint-Wise Dynamics. Bridging the sim-to-real gap is the heart of DexNDM (see Sec. 3.3 of the paper). The key idea is an independently-learnt neural dynamics model for the hand, factorized per joint. Why per joint? The whole hand–object system is very high-dimensional, so learning a single neural network to predict all 16 joint movements from full state histories is hopelessly data-hungry. Instead, Liu et al. “factorize” the problem: they train one small network per joint (per degree-of-freedom). Each joint-wise model predicts that joint’s next state (angle/velocity) from only that joint’s own recent history of states and actions, plus a learned pair of “effective” hidden variables. These hidden variables (not directly observed) distill the net effect of everything else: all other joints’ torques, object contact forces, etc., into a low-dimensional influence on this joint. In equations, each joint model is like. q_i(t+1) = f_ψ_i( q_i(t-W+1. t), a_i(t-W+1. t) , H^eff(t), G^eff(t) ),. where {q_i,a_i}^t_{t-W+1} is the window of the ith joint’s recent state-action history, and H^eff, G^eff are effective terms learned implicitly by the network. Crucially, every joint model is identical in form (sharing a network architecture), and there are 16 of them (one per joint). This joint-wise modeling creates an intentional information bottleneck. By only feeding each network its own joint’s signals and forcing it to compress all other influences into a tiny hidden state, the model is discouraged from overfitting to spurious correlations present in the limited data. As the authors put it, this “learns from a low-dimensional, information-contractive, task-sufficient representation of the system dynamics”. In practice, that means the model captures each finger’s true dynamical response (including object forces) separately. The result is a dynamics model that is surprisingly expressive yet highly sample-efficient. In our context, it means even thousands of real transitions (rather than millions) can suffice to learn an accurate predictive model of the real hand. By contrast, a monolithic “whole-hand” model (one network for all joints) proved much more data-hungry and even hurt performance in practice (more on that below). Why is this factorization sensible? Intuitively, if two joints only indirectly influence each other (e.g. through the object), then abstracting that influence avoids modeling every detail of the object-hand coupling. The authors even give a theoretical data-processing bound showing that applying such a low-dimensional projection can only reduce transfer error between domains (sim vs real). Empirically, they show each joint-only model, when trained on limited real data, generalizes much better than a larger model trained on the same data. The network simply has far fewer claims to learn, so each sample carries more force. In summary, the Joint-Wise Neural Dynamics Model (NDM) is at the core of DexNDM: it learns how the real robot hand moves differently than the sim hand, but in a split-per-joint, low-rate scheme. Once trained, it can predict the hand’s next state under real physics given a command, without any explicit object state. Autonomous Real-World Data Collection. A severe bottleneck in sim-to-real is obtaining enough real-world data for the dynamics model. Traditional methods – drop trials, random action sequences, or failed policy rollouts – either require constant resets/human intervention or result in mostly failed episodes. DexNDM sidesteps this by an ingenious autonomous data-collection strategy that requires no human operator and no successful rotations at all. The idea is simple: place the robot hand (with an object in grip) in continuous contact with unpredictable external forces, and record what happens as you replay simulation trajectories. Practically, they built a “Chaos Box” apparatus (Fig. 29 in appendix). This is literally a large box filled with many soft foam balls. The robot hand is inserted through the top (with the object held) and repeatedly “bathed” in the sea of balls. Because the balls are constantly shifting and gently colliding with the hand, each finger experiences a random, time-varying load. The system can run 24/7: they toss balls into the box, and the hand inside is always interacting. This produces huge diversity of contact forces, completely autonomously (they remark it can just collect data overnight with no human). Crucially, it works for palm-down, thumb-up, sideways and almost every orientation except full palm-up (the arm kinematics prevents reaching up inside the box). To cover palm-up cases, they use a smaller “Bandaged Ball” setup: a single soft ball is strapped to three fingers, and the arm moves and shakes it. This also operates without intervention (once set up), though with slightly less perturbation variety. In operation, they take a set of action trajectories (rollouts of the base sim policy) and replay them on the physical robot hand while it is loaded by balls or the bandaged ball. In other words, the hand executes simulated motions, but with all the real unexpected forces applied. Over many episodes this gathers a broad dataset of real transitions (states and the next states under real physics). In total, they collected on the order of thousands of episodes. For example, one regime was 4,000 replayed episodes using the Chaos Box, yielding about 1.6 million state-action transitions. (By contrast, only 20 actual successful rotations were collected — not for training, but as a performance check on a 5cm cube in an unusual orientation.) No delicate labeling or human resets were needed, and critically, the data generation did not require any demonstration of correct rotation. In fact the data is quite biased: it contains plenty of zero-motion or bouncing episodes. DexNDM learns from all of it. This stands in sharp contrast to “task-specific” data collection schemes: in DexNDM, they explicitly did not wait for the object to reach any goal orientation. The payoff is that the joint-wise NDM can be trained on this large, varied dataset. Because the data is already vast, cheap, and covers many contact patterns, the resulting model nicely generalizes to the rotation tasks at hand. Liu et al. stress that this autonomous scheme is efficient and general-purpose – “object-agnostic” – since it doesn’t even need vision or pose estimation in real time. We lose little if any relevant information, because the goal of data collection is not to fine-tune on specific desired rotations, but simply to let the model learn real dynamics. In ablation, they show that this “Chaos Box” method outperforms more naive schemes (like rolling out the policy or random actuations without controlled loads) in terms of the downstream performance of the trained model. Residual Policy Adaptation. With a joint-wise dynamics model (f_\psi) trained to predict real-world transitions, the final step is to adapt the sim-trained policy. DexNDM does this by learning a small residual policy π_res that adjusts the base policy’s actions. Conceptually, at each timestep the base policy wants to apply an action (a_t) (joint targets) given observation (o_t). DexNDM’s residual policy then outputs a corrective offset (\Delta a_t). The combined command executed on hardware is (a_t + \Delta a_t). The residual shifts the motion so that, under the real dynamics, the hand behaves as the sim policy intended. Formally, the authors minimize the difference between the next state predicted by (f_\psi) (which models real dynamics) and the sim base policy’s nominal next state. They set up an optimization: [ \min_{\pi_{\text{res}}} \mathbb{E}\Big|;q_{t+1} - f_\psi\big({q_{j},a_j + \pi_{\text{res}}(o_t, a_t)}{j=t-W+1.t}\big)\Big| ] where (q{t+1}) is the ground-truth next joint state from the simulation rollouts of the base policy. In practice, they solve this by supervised regression: on the dataset used to train π_base (simulation trajectories), they train π_res to predict the corrective action that would make (f_\psi) match those trajectories. At deployment, the robot simply adds these deltas to the base action. This residual adaptation is remarkably effective and simple to implement. Notably, they tried the obvious alternative of fine-tuning the entire policy network on the learned dynamics (using (f_\psi) as a pseudo-environment), but found it to be unstable and unreliable (small changes could ruin the learned behavior). The residual approach, by contrast, is decoupled and stable. The authors note that using π_res requires only “minimal specialized training techniques” compared to finetuning. In ablations, they see that including the residual is essential: without it, the policy can only rotate an object a fraction of a circle; with it, rotations can continue indefinitely in tests (Fig. of supplement shows nearly complete circles vs only 5 full rotations otherwise). Experimental Results. DexNDM’s claims are strikingly borne out in evaluation. The authors conduct an extensive suite of comparisons, both in simulation on unseen objects and on the real robot hand (the 16-DoF LEAP hand by Shaw et al. 2023). We highlight the headline findings:. Flexible Sim-to-Sim Generalization. First, they stress-test the generalist policy in the simulation on completely novel objects. This establishes a baseline that the BC distillation worked. Against a re-implementation of AnyRotate’s algorithm (as a representative prior method), DexNDM’s policy achieves far better rotation performance across axes. For example, when rotating along the ±x-axis with random wrist yaw, AnyRotate managed on average ~92 radians of rotation before falling, whereas their policy managed ~144 radians (and a higher success rate). Across the ±x, ±y, ±z axes and random orientations, the DexNDM sim-policy shows 30–80% better “success rotation” metrics, while also spinning longer and more stably (as measured by Time-To-Fall and steady rotation speed). In other words, even before any real-world tuning, their thinner, long-history-trained policy was already more adaptable than a leading baseline. Table 1 in the paper breaks down these multi-axis results quantitatively. Comparison to AnyRotate (Real Objects). More importantly, on real hardware DexNDM vastly outperforms baselines. They reproduce AnyRotate on four representative objects used in that work (a cube, a container, a tin cylinder, and a gum box) under the same gravity-invariant conditions. On each object, they report the total rotation angle (in radians) achieved before failure, and the time-to-fall. DexNDM outstrips AnyRotate by large margins. For instance, on the cube object, AnyRotate achieved about 6.5 rad, whereas DexNDM achieved 39.1 rad before dropping – roughly six times more. On the gum box, AnyRotate saw ~5.2 rad vs 13.96 rad for DexNDM; on a tin cylinder, ~2.6 rad vs 10.12 rad; on the container, ~3.7 vs 11.0 rad. Time-to-fall shows similar gaps. This is an enormous improvement: objects that AnyRotate could barely budge are spun many full turns by the DexNDM policy. The authors note that AnyRotate was only tested on moderately sized, simple shapes, whereas DexNDM tackles much smaller items and higher aspect ratios. Comparison to Visual Dexterity. DexNDM also matches or beats Visual Dexterity on the latter’s own test objects, despite VisualDex being a very strong benchmark. Because the tasks definitions differ (continuous rotation vs goal reorientation), they devised a “survival angle” metric: how many 90° increments the object is rotated before falling, inferred from the reported videos. Here DexNDM held its own — often doing as many or more spins. For example, on a bunny object, VisualDex can achieve about 5 full 90° turns, while DexNDM achieved 6 (roughly 6×90°). On many objects (cow, elephant, train, etc.), DexNDM matched VisualDex’s best. And remember, DexNDM also handles tiny and long items that VisualDex never attempted. In summary, DexNDM closed the gap with these specialized systems, despite starting from an entirely self-supervised sim-to-real procedure. Ablating Dynamics Models. To validate the joint-wise modeling choice, they compared against a “whole-hand NDM” that tries to predict all joints at once from the full state-action history. In simulation with abundant data this whole-hand model can be expressive, but in the low-data sim-to-real regime it fails catastrophically. In real-world tests, a policy adapted using the whole-hand model rotated only a few radians on hard orientations. For example, under a palm-down axis, the whole-hand NDM policy managed ~3.5 rad rotation (very little) compared to ~13.2 rad by DexNDM. In fact, the whole-hand variant often did worse than simple direct transfer. This confirms that the factorized model is the key to efficiency – it generalizes far better with the same limited data. Real-World Multi-Axis Rotation. Table 4 (left) of the paper summarizes rotation performance on hardware for a range of axes (±x, ±y, ±z) with the hand in the “palm-down” orientation. Here, the unadapted policy (“Direct Transfer”) could only manage moderate rotations (e.g. ~10.0 rad on z-axis) and sometimes dropped after ~20–25 seconds. With DexNDM adaptation, these numbers nearly double. E.g. at ±z axis, the direct policy rotated only ~7.6 rad, whereas DexNDM achieved 13.2 ± 1.7 rad on average; time-to-fall jumped from ~21 s to ~29 s. Similar lifts occur on x/y axes. Conversely, the whole-hand model’s line in the table shows practically no improvement (and poor rotation), again highlighting the necessity of the joint-wise model. Real-World Multi-Wrist Orientation. Table 5 shows results for rotation about the world’s z-axis while the robot’s wrist is in various orientations (Palm Up, Palm Down, Base Up/Down, Thumb Up/Down). These are very challenging because the fingers press in different directions or work against gravity. DexNDM shines here too. For example, with the palm-up pose (arm curled under), the raw policy did ~10.0 rad rotation (25.6 s TTF) but DexNDM did 14.6 ± 1.1 rad (32.8 s). In every pose, DexNDM out-rotated the raw policy, often by a factor of 1.5–3x. Notably, the hardest cases for other methods (“Base Up/Down”) saw especially large gains. Again, the whole-hand model baseline tracks near-zero in many cases. In short, DexNDM recovers much of the sim policy’s capability across wrist poses, whereas other approaches could not. Teleoperation & Task Transfer. Beyond pure rotation measurements, the authors demonstrate DexNDM in practical settings. They integrate the dexterous rotation policy into a VR teleoperation system. A human operator, wearing a Quest 3 controller, commands the hand via DexNDM’s policy. With this, they complete complex long-horizon tasks such as assembling a multi-legged mini-table, plugging in objects, using a screwdriver, etc. These tasks require fine reorientation of objects (screws, lids, tools) mid-air, which the DexNDM controller can perform robustly. This application underscores the practical utility: where an ordinary sim-trained policy would fail to, say, rotate a bottle to screw it in, the DexNDM-augmented policy succeeds. The paper shows video snapshots (Fig. 7, 19, 20) of these teleop scenarios, but even without metrics, it’s compelling real-world evidence that the learned policy is usable for robotics. These results paint a clear picture: DexNDM’s combination of broad sim training and clever sim-to-real bridging yields capabilities far beyond the prior art. It actually enables new possibilities – the robot can handle objects up to 20 cm long, down to 3 cm small, at AR up to 5.3: nothing like that had been shown for in-air rotation before. The data show that merely transferring the sim policy (Direct Transfer) often cannot even complete one full rotation on the hardest shapes, while DexNDM’s residual compensation makes those tasks routine. Significance and Discussion. DexNDM represents an important step forward for dexterous manipulation and sim-to-real algorithms. A few takeaways for the embodied AI and robotics communities:. Bridging Sim-to-Real with Neural Models: Historically, sim-to-real for dexterity meant either domain randomization or manual fine-tuning on real data. DexNDM demonstrates that a learned dynamic model can effectively close the gap, even for extremely contact-rich tasks. Its joint-wise factorization is a novel twist: it suggests that future sim-to-real models might embrace more such structured representations to handle partial observability and limited data. Learning from “Bad” Data: A striking aspect is that DexNDM’s real-world dataset is mostly biased and full of non-successful attempts. They didn’t require the robot to succeed in any rotation during data collection; in fact, most data is from the robot being buffeted by balls while following some policy. Yet, the model learned plenty. This echoes ideas from control theory and model learning that any rich excitation of the system can suffice, as long as the model looks at the right compressed signals. It’s a welcome reminder that collecting thousands of near-failure transitions can be more valuable than a few perfect demonstrations (which are expensive). Dramatically Expanded Object Set: With DexNDM, the robot’s effective “object repertoire” leaps to include long rods, flat books, small blocks, sculptural shapes – under orientations like base-up (hand thumb pointing down) that were previously impossible. This broad generality opens the door to using in-hand dexterity in everyday tasks (assembling furniture parts, orienting tools, etc). The fact they could generalize to an animal figurine or a rubber duck is impressive. It also validates the idea of distilling many category-specific policies into one: in sim the policy was already quite general, and the real-world adaptation preserved that. Comparison to Prior Work: DexNDM does not rely on vision for the rotation (only proprioceptive and contact sensing), and yet matches or outperforms vision-based planners like VisualDex in its tested scenarios. It also surpasses AnyRotate’s novel use of high-fidelity tactile sensors and sophisticated inner loop. This suggests that smart use of learning and residual control can complement or even replace expensive sensor setups for some tasks. Future systems might combine both ideas: a vision-guided target set by a task planner, and a DexNDM-like controller to execute precise reorientation. Some limitations remain. DexNDM’s pipeline requires an initial sim training and quite a bit of computation (they used thousands of simulated RL trajectories and millions of sim frames). It also relies on accessible real hardware (they used an Allegro-like LEAP hand and Isaac Gym sim). The chaotic data collection, while clever, needed a special setup. And as authors note, the model’s ultimate performance is still capped by the diversity of data it saw – if an object type or contact mode was never covered, it might fail. It also doesn’t directly handle agile insertion (like threading) where full pose precision is needed – it was all immeresed free rotations. But these are engineering issues that can be addressed in future work (e.g. more autonomous rigs, combining with vision for contact-rich tasks). For now, DexNDM provides a clear blueprint: train your dexterous policy broadly in sim, then learn how reality modifies it, and finally correct actions with a fast residual network. In doing so, they cracked one of the hardest nuts in robotic manipulation. The practical upshot is profound: robots can now perform a new class of in-hand tasks that was almost unthinkable before. We may soon see household or industrial robots routinely flipping objects in hand with the finesse of a juggler – all thanks to ideas like joint-wise neural dynamics and chaotic data collection. References: DexNDM (Liu et al., ICLR 2026); AnyRotate (Yang et al., 2024); Visual Dexterity (Chen et al., Science Robotics 2022).