Latest / Embodied AI 101 / The Sim-to-Real Gap Inside the Gearbox

Transcript

The Sim-to-Real Gap Inside the Gearbox · Embodied AI 101

• The Sim-to-Real Gap Inside the Gearbox. Requested paper: Enhancing Sim-to-Real Transfer for a High-Gear-Ratio Quadruped Robot via Extended Actuator Dynamics Identification • Hansol Kang et al. (as supplied) • Affiliations, venue, and publication year unverified. Source note — September 13, 2026: I could not retrieve the supplied DOI’s publisher page or locate a verifiable full text under the supplied title. Consequently, this is a research-backed companion episode about the technical problem—not a reconstruction of Kang et al.’s methods or results. Findings below are attributed to accessible primary sources; the proposed workflow and evaluation criteria are our engineering synthesis. A PDF or corrected citation is needed for a paper-specific review. When a good policy meets the wrong actuator. Imagine a quadruped policy that looks entirely convincing in simulation. It tracks velocity commands, rejects pushes, and produces clean foot clearance. On hardware, however, the robot hesitates near joint reversals. A stance leg yields differently than expected. Increasing position gains improves one behavior while making another more violent. Nothing in this scenario necessarily implicates perception, policy capacity, or the reinforcement-learning algorithm. The policy may simply have learned to control an actuator that does not exist. That possibility motivates the question behind the supplied title: how much of the actuator’s real behavior must enter simulation before locomotion training becomes trustworthy? This is an established concern. In Sim-to-Real: Learning Agile Locomotion for Quadruped Robots, Jie Tan and colleagues explicitly combined system identification, actuator modeling, and latency simulation with robustness-oriented training. Accurate actuation and robust learning were complementary ingredients, not competing explanations for successful transfer. For a highly geared robot, the interesting question is more specific than whether the simulator has the correct maximum torque. We need to know what happens between a command and the resulting joint motion, particularly when the environment pushes back. Does the joint start moving immediately? Does resistance change under load? Does the transmission behave differently when the motor drives the leg and when ground contact drives the motor? Are the same apparent tracking errors produced by inertia, friction, delayed commands, or electrical limits?. Those distinctions determine what to identify. Our central argument is therefore methodological: before enlarging the policy or widening every randomization range, determine whether the simulator contains the mechanisms responsible for the observed mismatch. Randomizing the parameters of an inadequate model can still leave the real actuator outside the family of behaviors the policy experiences. The useful outcome is not necessarily a more elaborate simulator. It is a simulator whose remaining errors are understood well enough to train against. Why gearing changes more than torque capacity. An ideal reduction gear exchanges rotational speed for torque. It does not create mechanical power. That elementary relationship is easy to remember and surprisingly easy to violate when configuring a simulated actuator with independent torque and velocity limits. A useful model must describe the operating envelope, not just two favorable numbers copied from a specification sheet. The less intuitive effect is reflected inertia. In the usual output-side lumped model, motor rotor inertia contributes an amount multiplied by the square of the reduction ratio. Holding the motor fixed, doubling the reduction therefore quadruples this rotor contribution—not the entire robot’s inertia, but the component associated with accelerating the rotor through the transmission. MuJoCo’s documentation describes this explicitly through its joint parameter. Consider the modeling consequence. If a simulated leg contains the link inertia but omits a substantial reflected rotor contribution, its joint can accelerate and reverse too easily. An optimizer may then “repair” the trajectory mismatch by weakening the motor or increasing damping. That is a plausible fit to one experiment, but it is not necessarily the right physical explanation. Friction introduces another distinction. Linear damping opposes velocity; it cannot, by itself, represent a joint that remains stationary while resisting a finite applied torque. MuJoCo handles its load-independent dry-friction term as a bounded constraint force rather than merely a velocity-dependent force. That distinction matters precisely around the stopping and reversal events that a smooth trajectory plot can conceal. Transmission losses also need not be symmetric between driving and backdriving. These are directions of power transmission, not simply positive and negative joint rotation. A motor pushing against an external load and an external load forcing the motor backward can occupy different regimes. Masuda and Takahashi explicitly model this directional transmission efficiency in their work on a torque-sensorless, gear-driven humanoid. Finally, an actuator has temporal behavior. Software and hardware feedback loops contain states that may not appear in the robot’s usual observation vector. Hwangbo and colleagues’ actuator-network formulation addressed this by predicting joint torque from histories of position errors and velocities, rather than treating the current command as sufficient. The engineering implication is a diagnostic one. A sluggish joint does not uniquely imply excessive damping. A delayed reversal does not uniquely imply communication latency. Several mechanisms can create similar-looking traces while responding differently to load, frequency, and gain changes. A useful identification campaign deliberately creates those changes. Otherwise, it risks producing a numerical explanation that works only for the calibration motion. Define the interface before identifying its dynamics. Before selecting an optimizer, I would write down the complete command path. Suppose the policy outputs desired joint positions. Those targets pass through scaling, clipping, communication, a position controller, possibly a velocity controller, and an electrical control loop before producing motion. Where, exactly, does the simulated actuator model begin? Which of those transformations already exist elsewhere in the simulator?. This is not documentation housekeeping. It determines what the fitted parameters mean. If the model receives desired position and predicts applied torque, it represents a closed-loop actuator, including whatever controller behavior lies inside that boundary. If it receives commanded motor current, the position controller belongs outside the model. Fitting one interface and deploying another would change the object being identified. The same audit is necessary on the measurement side. A log field named “torque” could mean requested torque, current-derived motor torque, or measured output torque. These are not interchangeable labels. Fey and colleagues’ Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation specifically identifies the limitations of current-derived torque labels for actuators with nonlinear harmonic-reducer behavior. I would therefore make every recorded signal carry three pieces of metadata: where it was measured, what transformation produced it, and when it became available. For encoders, record whether the measurement is motor-side or output-side. For velocity, record whether it is directly reported or numerically estimated, and what filtering is applied. For commands, distinguish the policy’s intended output from the packet actually transmitted and, where observable, the command accepted by the drive. Timing deserves the same treatment as units. A model can appear impressively accurate if command and response channels are aligned using information that would not be available during simulation rollout. My implementation audit would explicitly check that every model input is causal. There is also a distinction between simulator information and policy information. An actuator model inside a simulator may use internal simulated quantities to implement a physical mechanism. That does not automatically justify exposing those quantities to the deployable actor. The model interface and policy observation interface should be reviewed separately. The deliverable from this stage is a signal diagram and a timing contract. Without them, a fitted “motor parameter” may actually be compensating for preprocessing, gain conventions, or a one-sample offset. What should an “extended” model have to earn?. Without the requested full text, we cannot know what “extended” denotes in Kang et al.’s formulation. It might refer to additional physical effects, a broader identification procedure, or another change entirely. For our own implementation, I would make extensions compete against a nested sequence of models. Start with the controller that the hardware actually executes, together with a defensible torque-speed envelope. Then add reflected inertia and measured timing behavior. Next, introduce the friction mechanisms required by the data. Only after testing those components would I consider a learned residual or additional internal states. This sequence is a proposal for diagnosis, not a universal ordering. Its purpose is to make each increment answer a specific question. For example, does adding inertia improve high-acceleration segments without corrupting slow motion? Does a dry-friction mechanism improve stopping behavior across different gains? Does a load-sensitive term explain errors that appear only when external resistance changes?. A model should earn its complexity by explaining a pattern that a simpler model systematically misses. There is an implementation subtlety here. Load-dependent friction can require information about external torque, while external torque and friction are themselves coupled through the dynamics solve. Duclusaud and colleagues discuss this explicitly in Extended Friction Models for the Physics Simulation of Servo Actuators, including using the previous external-torque estimate when updating the friction model. This is not merely a regression problem; it also concerns how the model participates in numerical integration. For a learned residual, I would ask another question: what is the residual allowed to repair?. A correction to commanded torque, a correction to output torque, and a correction to joint state are different modeling choices. They change interpretability, constraints, and how errors propagate. I would prefer an interface whose failures can be inspected in the same units as the mechanism being modeled. More parameters are not inherently better. A compact model that survives a load change is more useful than an elaborate model that memorizes the original trajectory. Collect experiments that separate explanations. A concrete precedent comes from Bjelonic, Tischhauser, and Hutter’s PACE work, Towards bridging the gap: Systematic sim-to-real transfer for diverse legged robots. Their pipeline collects fixed-base, in-air trajectories, aligns a compact simulator parameterization, and then trains locomotion policies. The reported excitation uses PD-tracked chirps, with sequences lasting 20 to 60 seconds. This makes actuator calibration possible before a reliable locomotion controller exists. That is an attractive starting point, but I would not treat one chirp as a complete identification experiment. Instead, I would organize data collection around competing explanations. Slow motion and sustained positions help probe low-speed behavior. More rapid changes challenge acceleration response. Repeated reversals expose differences between approaching zero velocity and leaving it. Changing the external load tests whether a model learned in one loading condition remains explanatory. These should be separate experimental dimensions wherever practical. If amplitude, frequency, and load all increase together, a later improvement in fitting error will be harder to interpret. I would also repeat representative tests at the gains intended for deployment. A calibration performed only under an unusually stiff tracking controller answers a different question from one performed under the compliant settings used by the locomotion policy. The objective is not to make the robot track every excitation perfectly. Tracking error is part of the evidence. The objective is to apply reproducible references while observing the actuator in relevant operating regimes. For initial hardware work, my proposed sequence would begin with secured, low-amplitude motion and explicit current, temperature, and travel limits. Fixture behavior should be recorded rather than assumed irrelevant. If the nominally fixed base moves, or the support adds compliance, the simulated experiment should account for that condition or the dataset should be excluded. Validation data should be designed before fitting. I would reserve complete trajectories, not merely random samples from the same recording. I would also reserve at least one meaningful change of condition: a different load, gain setting, excitation family, or hardware session. That gives the validation result a clear interpretation. The model is being asked to explain a new experiment, not adjacent samples from the experiment it already saw. The minimum useful dataset description therefore includes commands, measurements, timing, gains, limits, loads, mounting conditions, and the train–validation split. “We collected several minutes of motion” is not enough to assess what those minutes identify. Fit an experiment, not just an error scalar. Suppose we now have synchronized reference trajectories and measurements. One reasonable fitting strategy is to replay each reference through the simulated controller and minimize disagreement with measured joint motion. I would begin with position error because it is directly observable and easy to inspect. Velocity error can add useful discrimination, but only after checking the velocity signal’s construction. A derivative estimated from noisy position data should not silently become the most heavily weighted target. I would also inspect errors by regime rather than relying exclusively on one aggregate score. Does the model miss the phase of the motion, its amplitude, or both? Does it fail mainly at reversals? Does error grow with load? Does it consistently predict movement before the hardware begins moving? Each pattern suggests a different next experiment. The fitting algorithm should support this reasoning, not replace it. An important trap is parameter tradeoff. A weaker actuator and a larger effective inertia can both slow acceleration. A delay and a slower dynamic response can both shift a trajectory. Over a limited operating range, different parameter combinations may look similarly plausible. I would not interpret a fitted value as a measured physical constant unless the experiment actually supports that interpretation. Sometimes the appropriate result is an effective parameterization that predicts the tested behavior, with uncertainty about its internal decomposition. A useful stress test is to add independent information. If the external inertia is changed by a known amount, does the model predict the new response without re-identifying everything? If a controller gain is changed deliberately, does the model still behave correctly?. These interventions are more informative than another decimal place in the training loss. I would also compare one-step and rollout behavior. A model can make small local errors yet drift in a recursively simulated trajectory. Conversely, a long rollout can accumulate phase differences that obscure whether the local actuator law is reasonable. Both views deserve inspection. Finally, fitted parameters should be tested across numerical settings. If a seemingly physical friction coefficient must change dramatically whenever the simulation timestep changes, that is a reason to investigate the discretization or model coupling before drawing hardware conclusions. The result we want is not simply the smallest scalar loss. It is a reproducible explanation with a stated operating envelope. What the neighboring literature actually establishes. Several accessible studies clarify the design space. They should be read as complementary examples, not as a single leaderboard. In Learning Agile and Dynamic Motor Skills for Legged Robots, Hwangbo and colleagues trained an actuator network using real data and incorporated it into simulation for ANYmal policy learning. The central modeling choice is a history-dependent mapping from position errors and velocities to torque. This provides a practical alternative when a detailed analytical description of the actuator and its internal control loops is difficult to obtain. It does not imply that every robot needs the same network or history length. Duclusaud and colleagues take a more explicitly physical route. Extended Friction Models for the Physics Simulation of Servo Actuators investigates friction beyond a basic Coulomb-viscous description, identifies models using pendulum experiments, and evaluates four servo actuators and two-joint manipulators. The reported improvement concerns actuator simulation accuracy. It is relevant evidence for richer friction modeling, but it is not itself a quadruped locomotion-transfer result. Masuda and Takahashi connect transmission modeling more directly to legged control. Their Sim-to-Real Transfer of Compliant Bipedal Locomotion on Torque Sensor-Less Gear-Driven Humanoid combines a gear model with an iterative identification process on ROBOTIS-OP3. After initial identification and policy transfer, failed attempts can inform re-identification through differences in acquired rewards. The important distinction is that this later stage uses a task-level measure of behavioral similarity, rather than requiring accurate long-horizon trajectory matching. Fey and colleagues’ Unsupervised Actuator Net offers another route when reliable torque labels are unavailable. UAN learns corrective simulator torques from discrepancies between simulated and real joint behavior. Their project also illustrates a particularly useful warning: a recorded open-loop motion can look reasonable on hardware while the corresponding closed-loop policy fails. Agreement under command replay is therefore not sufficient evidence of policy transfer. A different intervention appears in Yamamori and colleagues’ July 2, 2026 preprint, Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning. Instead of making simulation reproduce all hardware behavior, the authors shape the physical actuator’s closed-loop response toward prescribed second-order reference dynamics. Their feedforward–feedback architecture is evaluated through single-joint experiments and a seven-degree-of-freedom arm task, with additional locomotion demonstrations. The conceptual contrast is valuable: sometimes the interface can be standardized on hardware rather than modeled in increasing detail. These approaches answer different questions. Do we have trustworthy torque measurements? Can a compact physical model explain the relevant behavior? Can we modify the low-level controller? Must calibration work before stable walking? Is the objective accurate joint dynamics, successful task behavior, or both?. That is how I would choose among them. A favorable result on one platform does not settle the choice for another transmission, sensing configuration, or control interface. A calibrated actuator still needs honest reinforcement learning. Once the actuator model is fitted, it is tempting to declare the sim-to-real problem solved. I would instead treat calibration as the construction of a better nominal training environment. Robustness still needs deliberate testing. Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization explores random force injection and an extension that adds an episode-level actuation offset. Its demonstrations on ANYmal C and Unitree A1 show that structured actuation perturbations can be effective without an exhaustive catalog of randomized physical parameters. That is evidence for a useful robustness tool, not proof that identification is unnecessary on every robot. My proposed combination is straightforward: identify the reproducible behavior, then randomize what remains uncertain or genuinely variable. The resulting randomization should have an interpretation. A persistent calibration offset is not the same uncertainty as fast command noise. Variation between actuator units is not the same as variation within one control cycle. I would encode these distinctions explicitly rather than giving every parameter an independent sample at every step. The training implementation also needs to preserve the identified timing contract. As an illustrative example, a policy updating at 50 hertz and a servo updating at 1 kilohertz create twenty servo updates per policy action. Whether the target is held, interpolated, filtered, or delayed during those updates belongs in the simulated interface. I would test that behavior independently before training a walking policy. The same applies to action scaling, clipping, reset behavior, and initialization of model history. Another review point is operating coverage. A locomotion optimizer may discover commands unlike the calibration trajectories. If it repeatedly drives the actuator model into untested regions, the right response is not automatically to trust the learned behavior. I would first inspect those regions and decide whether to constrain them or collect additional controlled data. Calibration should therefore produce two artifacts: a model and a description of where it has been tested. The second artifact is what lets the learning team distinguish an ambitious new behavior from exploitation of an unvalidated approximation. What convincing evidence would look like. For a paper claiming improved transfer through extended actuator identification, I would separate three questions. First, does the extension improve prediction? Second, does it improve learned control on hardware? Third, can the improvement reasonably be attributed to the extension rather than another change in the pipeline?. The first question calls for held-out actuator experiments. I would want errors broken down by joint, direction, load, speed, and motion regime. An average over all samples can be dominated by easy sections while concealing the events that matter most for contact transitions. The second question requires closed-loop hardware evaluation. My proposed protocol would use a predefined command set and report tracking performance, failure or intervention frequency, and behavior under specified disturbances. If energy efficiency is claimed, the measurement boundary should be stated explicitly rather than inferred from a torque penalty. The third question is where ablations become essential. I would compare a simpler actuator model and the extended model under otherwise matched training conditions. If domain randomization is also changed, I would test the combinations rather than allowing two improvements to arrive as one inseparable package. Policy seeds matter here. So do repeated hardware trials. One successful transfer demonstrates possibility; repeated trials help characterize reliability. I would report the trial order and reset procedure, especially when actuator condition may change over the course of testing. A fair comparison should also preserve the real control interface. If one method is deployed with different gains, stronger filtering, or more restrictive action limits, that difference is part of the method and should be visible. Then there is the accounting of engineering cost. How much hardware data was required? How many identification attempts failed? Was a dedicated fixture necessary? How long did optimization take? How much simulator throughput was lost?. These costs can determine whether a theoretically attractive model is useful in a working locomotion pipeline. Finally, I would distinguish platform-specific and cross-platform claims. An extension that works well on one robot is valuable. Demonstrating that it transfers across actuator units, payloads, or transmission designs is a separate result requiring separate evidence. The standard is not perfection. It is a chain of evidence connecting a diagnosed mismatch, an identifiable model change, improved prediction, and improved hardware behavior. Putting the idea to work. For a team facing a transfer failure now, I would start with the smallest experiment capable of separating two plausible explanations. If the error appears mainly during reversals, compare those events across loads and gains. If it appears mainly during fast motion, compare timing and acceleration response before adding a complicated friction law. If free-space behavior agrees but loaded behavior does not, make external loading the next experimental variable. Do not begin by asking which actuator model is fashionable. Begin by asking what observation would make the current explanation wrong. For implementation resources, PACE’s public documentation is a verified entry point with installation material, basic usage, and an ANYmal example. It provides something concrete to inspect when designing an identification pipeline, without implying that its assumptions automatically match a different robot. I would keep calibration replay as a regression test alongside the locomotion code. Whenever gains, firmware, timing, simulator settings, or actuator configuration change, rerun the same small suite. That makes it possible to detect an altered action-to-motion mapping before spending another training budget on it. The larger lesson is not that every highly geared quadruped needs a digital twin with every loss mechanism modeled explicitly. Our preferred target is narrower: enough structure to explain the important behavior, enough experimentation to distinguish the parameters, and enough validation to know where the explanation stops. The actuator is part of the learning problem’s dynamics, not a detail to fix after training. Treating it that way changes how we collect data, construct simulations, interpret transfer failures, and judge claims of progress. The research search established several strong precedents for this approach, but it did not establish the supplied Kang et al. paper’s exact model, experiments, results, or code. Those remain unverified; a full text or corrected citation is necessary to complete that specific analysis.