Latest / Embodied AI 101 / ANY2ANY: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking
Transcript
- ANY2ANY: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking. Humanoid robots have recently seen a surge of interest in learning-based whole-body control. Rather than hand-crafting separate controllers for walking, running, crawling and so on, modern approaches train a single “behavior foundation model” that can produce all these motions. A prime example is NVIDIA’s SONIC (“Supersizing Motion Tracking”) model, which uses large-scale human motion data to teach a robot a core set of motor skills. In SONIC, a human demonstrates movements (often via VR teleoperation) and the humanoid learns to track those motions. As the SONIC project page explains, this motion‐tracking approach enables a unified policy to execute natural, whole‐body behaviors like walking, dancing, or complex tasks, instead of piecing together separate controllers for each motion. This “foundation‐model” strategy has dramatically improved humanoid capabilities – SONIC and its kin can handle a wide range of whole-body tasks. But there is a catch: these high-performance policies are typically trained for one specific robot body. A SONIC policy trained on, say, a Unitree G1 humanoid will not directly work if you build a new robot with different limb lengths, joint ranges, or mass distribution. In practice, transferring a policy to a new embodiment usually means retraining or fine-tuning, which has proven extremely expensive. In the SONIC paper itself, the authors noted that training their model from scratch took “thousands of GPU hours and hundreds of millions of motion frames”. Even fine-tuning an existing policy on a new robot can take a similarly large effort and risks catastrophic forgetting of the original skills. In other words, naively “just retrain” for a new humanoid is effectively like starting over: it burns enormous compute, data and time, and may even destroy what was learned before. The ANY2ANY method, introduced by Yang et al. (LimX Dynamics) in 2026, tackles exactly this problem of cross-embodiment transfer. The key idea is to decouple the differences between robots into two parts – one due to kinematic structure, and one due to dynamics – and address each part separately. In their own words, ANY2ANY “decouples the transfer problem into two distinct stages.” The general skill (for example, the pattern of walking) is largely transferable between robots, but the details of how that skill is executed (e.g. due to different joint layouts, masses, and motor properties) are body-specific. ANY2ANY’s framework reflects this insight with a modular pipeline: first align the kinematics between the source and target robots, then adapt the dynamics of the policy with only minimal tuning. Decoupling Kinematics and Dynamics. The ANY2ANY transfer pipeline operates in two main stages. First comes Kinematic Alignment, which handles all the structural differences between the robots (joint counts, limb lengths, DOF ordering, etc.) outside the neural network. In practice this means adding an external “translator” around the policy network: when the new robot collects its sensor data, the alignment module reorders and possibly scales those inputs so that they look like the source robot’s sensors. The core policy (which was trained on the source robot) then computes an action as if it were still engaging the source body. Finally, a symmetric mapping takes those “virtual source‐robot” actions and converts them into real motor commands for the new robot. In effect, the policy network always thinks it’s controlling the original robot, even though the outer kinematic adapter is converting everything to and from the actual body. This idea is perhaps best understood with an example. Imagine the source robot and the new robot have the same number of joints but in a different order – say the arms are indexed differently. The kinematic alignment can simply permute the target’s joint-angle vectors into the ordering expected by the policy, and then invert that permutation on the output commands. If the target has more or fewer joints, the alignment mapping can drop or pad entries appropriately. In more complex cases, one can define analytical relationships between the screw axes or link frames of the two robots. The core point is that all the geometric differences are absorbed by a preprocessing and postprocessing layer, rather than by changing the neural controller itself. As Yang et al. explain, this kinematic alignment module uses “defined mathematical mappings to re-order and adjust the sensor data from the new target robot so it looks like it came from the original source robot,” and then reverses that mapping on the outputs. In short, the first stage makes the new robot kinematically resemble the source robot as far as the policy is concerned. With the kinematic stage in place, the problem is now strictly about dynamics – things like different masses, inertias, friction coefficients, actuator speeds, etc. At this point ALL the geometric mismatches are gone, and only the physical response differences remain. ANY2ANY addresses these with a lightweight, gradient-based adaptation of the policy using Low-Rank Adaptation (LoRA). LoRA, originally popularized for fine‐tuning large language models, inserts small trainable matrices into selected layers of the network. In ANY2ANY, the pre-trained policy network is “mostly frozen” – only these tiny LoRA adapters are allowed to change. Concretely, one might add a low-rank update to each weight matrix in the policy’s layers, so that the new robot’s data can tweak the motor outputs via these added parameters. The beauty of LoRA is that it introduces only a very small number of extra parameters. ANY2ANY’s authors highlight that these adapters account for “maybe one to five percent” of the original model’s weights. In other words, instead of fine-tuning millions of weights, we only train a few percent of them. This is a huge reduction in complexity. The intuition is that after kinematic alignment, the remaining mismatch in dynamics can often be represented by a low-dimensional corrective offset. The authors note that once the heavy lifting of kinematics is done, “the key insight is that the dynamic mismatch… can be represented as a low-dimensional correction”, and LoRA is well-suited to learn exactly those low-rank updates. Roughly speaking, the LoRA layers adjust how the policy interprets proprioceptive feedback and how it generates final torques, so that the robot’s own mass and inertia are taken into account. Crucially, training only the LoRA modules is far cheaper than full retraining. Yang et al. show that this lightweight adaptation converges much faster, with far less data, than training a new policy from scratch. As they emphasize, the policy with LoRA “converges far faster and with less data than training a specialist model from scratch”. In practice, this means one can reconfigure a whole-body controller for a new robot using only a small fraction of the original training experience. While the SONIC foundation model needed millions of frames and gargantuan compute, ANY2ANY’s adaptation can succeed with on the order of 1% of that effort (the authors report needing only ∼1–5% of the parameters and implying a similarly small fraction of data). Effectively, by freezing most of the network and only tweaking a few percent of weights, we get the new robot behaving almost as capably as if it had been trained extensively from scratch. In summary, ANY2ANY’s two-stage approach – first align kinematics outside the network, then adapt dynamics via LoRA – neatly addresses the cross-embodiment issue. The policy core remains largely intact, letting us reuse pre-trained motor skills wholesale; the outer adapters handle the robot differences. As the authors succinctly conclude, their framework “allows for the efficient reuse of powerful, pre-trained motor skills across different humanoid bodies”. Method Details: Kinematic Alignment and LoRA Adaptation. Delving into more detail, the kinematic alignment stage can be thought of as a custom mapping layer. It might be implemented, for example, as a fixed function that converts the new robot’s joint angles and velocities into a “canonical” representation that the source policy expects. Concretely, if the source policy was trained on the Unitree G1 humanoid (which has 31–43 joints depending on configuration), then we build a mapping from the LimX robot’s 27–31 joints into the G1’s joint space. During execution, the new robot’s state (joint encoders, base IMU, etc.) is passed through this mapping. The pre-trained policy takes this mapped state as input and outputs an action in the source robot’s frame. Finally, a reverse mapping converts that action (e.g. desired torques or joint targets for each G1 joint) into torques for the corresponding LimX joints. Because the mapping is defined by the engineer (or derived from kinematic models), it automatically handles mismatches in linkage. For example, if the source head has 2 degrees of freedom and the target head has the same 2 DoF, the mapping triviality pairs them. If one robot has an extra gripper or waist joint that the other does not, the extra output might be ignored or handled by a default rule. If link lengths differ, one could normalize joint angles by ratio of lengths. The ANY2ANY paper does not dwell on these low-level robotics details, but it clearly treats them as solved by mathematics outside the network. This means the neural policy itself never has to see a “mismatched” joint – everything always looks structurally correct to it. Once the structural alignment is in place, the remaining adaptation occurs entirely via the LoRA modules. Technically, LoRA works by inserting additional matrices into the policy’s linear layers. If a layer originally had weight matrix W, LoRA would replace it with W + ΔW, where ΔW is factored as A·B (two small matrices). Yang et al. do not specify exactly which layers they modified, but typically one would attach LoRA adapters to the larger linear layers (e.g. in the policy’s fully-connected layers). The important point is that the original weights W stay frozen at their pre-trained values; only the low-rank factors (A, B) are trainable. This means only rank(W) ≪ rank(W) fraction of parameters change. In practice, Yang et al. report using LoRA adapters that amount to only a few percent of the policy’s total parameters. Why is this enough? Intuitively, after kinematic alignment the core policy is already producing nearly correct motions – it “knows how to walk,” for instance – but on the wrong body it will produce, say, slightly incorrect torque to reach a reference pose. The small LoRA modifiers act like fine-tuners that adjust those torques just enough. As John explains in the transcript, “once the kinematics are aligned, the remaining error is purely due to dynamics – differences in mass, inertia, actuator response, and so on”. By training the LoRA parameters, ANY2ANY effectively learns those dynamic corrections. According to the authors, this approach works surprisingly well: because the dynamic mismatch really is just a small correction, LoRA (which is designed for low-dimensional updates) can learn it with ease. Even Noah – who initially questions “why LoRA specifically, not other methods?” – observes that empirical ablations found LoRA gave the best combination of performance and stability. In practice, training proceeds much like standard reinforcement learning or imitation on the new robot, except that all gradients flow only into the LoRA modules (and the kinematic adapter is fixed). One would collect some data on the target robot performing motion-tracking tasks and use it to update the LoRA parameters. Because there are so few of them, the optimization converges very quickly. The result is a new policy that achieves almost the same tracking behavior on the target robot as the original policy did on its robot, yet it required only a tiny budget of extra training. Experiments and Results. Yang et al. test ANY2ANY by transferring from a Unitree G1 humanoid (the source of the SONIC foundation policy, nicknamed “Gear-SONIC”) to two different LimX humanoids: LimX Oli and LimX Luna. Both LimX robots are full-size bipeds roughly 1.6 m tall and ~55 kg, comparable in stature to the G1. For reference, the LimX Oli has 31 degrees of freedom and LimX Luna is similarly sized (165 cm tall, 55 kg). These are serious humanoids with legs, arms, head, etc., so transferring the whole-body policy is nontrivial. The experiments (conducted in simulation, with the LimX models sufficiently accurate) show that ANY2ANY can indeed adapt the Gear-SONIC policy to both targets efficiently. While the paper does not list raw error plots in the excerpt we have, the authors report strong whole-body tracking performance. In qualitative demos, the adapted policies enable the LimX robots to reproduce the same kind of rich behaviors that the SONIC policy could do on G1: walking in various styles, crawling, jumping, bimanual manipulation, and even stylized motions like “happy” or “stealthy” walking. Crucially, this is achieved with dramatically less training effort. The authors highlight that they needed only on the order of 1% of the compute/data that the original Gear-SONIC training used, yet the LimX robots learned to track motions nearly as well as if we had trained a new policy from scratch. For concrete numbers: if the SONIC training took, say, 2000 GPU-hours and hundreds of millions of frames, ANY2ANY achieves a comparable result in the low tens of GPU-hours using roughly only a couple million frames (on the order of 1% of the original). This speedup comes from both the kinematic alignment (no need to re-learn the geometry) and the small parameter count of LoRA. In learning curves the policy loss reaches maturity very quickly. In fact, Yang et al. emphasize that the LoRA-based adaptation alone “converges far faster and with less data than training a specialist model from scratch”. In practical terms this means an adaptation run might take perhaps an hour of training vs days, and require maybe 10^5 frames of motion data instead of 10^7. The end result is that the LimX humanoids can perform rich motion tracking under the reused policy. ANY2ANY thus succeeds in its goal: to take a high-fidelity policy trained on one embodiment and port it, with minimal effort, to another. The LimX robots effectively inherit the Gear-SONIC skills even though their body mass and joint configuration differ. This validates the approach of splitting kinematics and dynamics: most of the walking skill was reused untouched, and only trivial corrections were learned. Implications for Robotics. The implications of ANY2ANY are significant for humanoid robotics development. Current practice often involves designing a new robot hand-in-hand with a new controller – but training advanced, general controllers is extremely costly. ANY2ANY offers a practical alternative: one can invest the massive compute effort in building one top-tier policy (a foundation model) and then cheaply port it to any new hardware iteration. The computation need not be repeated from scratch. As the authors note, this effectively shifts the paradigm from “training one model for all” to “efficiently adapting one model for any”. In industry terms, this could dramatically shorten development cycles. Imagine building a second-generation humanoid: instead of hiring another team of RL engineers to re-train a motion policy for that exact robot, you could simply apply ANY2ANY on an existing policy. The new robot would gain high-level behaviors almost immediately, only fine-tuned via LoRA. It lowers the barrier to incorporating learning-based control into new designs. Even small labs could benefit: they could license or adopt a publicly available foundation policy and adapt it for their own platform with minimal resources, rather than needing their own GPU cluster. ANY2ANY also complements – rather than replaces – larger trends in embodied AI. Some teams strive to build universal humanoid controllers by training on many bodies from the start. That is still valuable: a multi-robot foundation model could provide even better general skills to transfer. ANY2ANY could then serve as the “last mile” adapter, fine-tuning a generalist to a specific body. In that sense, ANY2ANY acknowledges the merits of foundation models (like SONIC or hypothetical multimodal controllers) but provides a practical means to reuse them. As the authors put it, ANY2ANY and pure multi-embodiment training are “two sides of the same coin”: you can either try to train one giant policy for all robots, or train one and quickly adapt it to any new one. Of course, ANY2ANY is currently demonstrated in simulation and on humanoids that are not wildly different (the LimX robots are roughly the same scale as the source). The kinematic alignment step relies on knowing a sensible mapping between joints – so if you jumped from a humanoid to a radically different robot (say a quadruped), you would need to design an appropriate mapping or face limitations. But for the common case of humanoid variants, ANY2ANY fills a crucial gap. It essentially says: “if you can walk on one biped, you can walk on a similar biped – just retune for the new anatomy.”. In summary, ANY2ANY provides a structured, efficient recipe for cross-robot transfer in whole-body control. By separating geometric from dynamic factors, it allows a powerful pre-trained policy to be reused across embodiments with minimal new training. As the authors conclude, this lets robotics practitioners “reuse the vast computational effort invested in training one expert policy across many different platforms”. In practical terms, it transforms what used to be a project requiring thousands of GPUs and months of data into something achievable with a single GPU and a few days of work. For lifelong robotics development, that is an extremely welcome improvement. References: Yang et al. (2026), Any2Any: Efficient Cross-Embodiment Transfer for Humanoid Whole-Body Tracking. (Also see NVIDIA’s SONIC foundation model overview.).