Latest / Embodied AI 101 / Feeling the World Without a Force Sensor: UniFP for Legged Loco-Manipulation
Transcript
- Feeling the World Without a Force Sensor: UniFP for Legged Loco-Manipulation. Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation • Peiyuan Zhi, Peiyang Li, Jianqin Yin, Baoxiong Jia, Siyuan Huang • BIGAI; Joint Laboratory of Embodied AI and Humanoid Robots, BIGAI & Unitree Robotics; Beijing University of Posts and Telecommunications • CoRL 2025 Best Paper • 2025. The missing half of robot action. A robot can know where its hand is and still have no idea whether it is meaningfully touching the world. That distinction is easy to overlook in modern robot-learning pipelines. We record camera images, joint states, and end-effector trajectories; train a policy to reproduce those trajectories; and rely on a position servo to turn predictions into motion. For free-space reaching and pick-and-place, this can work remarkably well. But once success depends on maintaining pressure, overcoming a latch, yielding to a person, or detecting contact under occlusion, geometric trajectories stop being a complete description of the task. Consider wiping a board. The tangential motion is visible and straightforward to demonstrate, but success also depends on an unobserved normal force. Too little pressure leaves the ink untouched; too much can damage the tool or destabilize the robot. A push-to-open cabinet presents the opposite problem: the important event may be a few millimeters of motion accompanied by a decisive change in reaction force. Vision can barely resolve the displacement, yet the robot must know when the mechanism has fired. UniFP attacks this gap at the control interface. Rather than treating force as an accidental consequence of executing a position trajectory, the policy accepts both position and force commands. At the same time, it learns to estimate external forces from a history of proprioceptive state—without a dedicated wrist force/torque sensor—and uses that estimate to compensate its motion. The result is one low-level whole-body policy that can behave as a position controller, a force controller, an impedance controller, or a hybrid of these modes. The same internal force estimate is then exposed to a Diffusion Policy that learns contact-rich tasks from demonstrations. The important contribution is therefore not a new law of force control. Hybrid force-position control and impedance control are decades old. The contribution is to turn those ideas into a learned, command-conditioned whole-body interface for legged robots—and then make the learned force representation useful to the rest of the robot-learning stack. What “unified” actually means. The conceptual starting point is an impedance model: displacement from a desired pose produces force according to a virtual stiffness. UniFP simplifies that model for slow, quasi-static manipulation by dropping the end-effector’s acceleration and velocity terms. In plain language, a force mismatch becomes a position offset. If the robot should apply more force, it moves farther into the environment. If an external force pushes the robot, the target moves in the direction of that push, allowing the robot to yield. The authors divide the interaction into three force components. There is the force the robot is commanded to apply, the reaction produced by the environment, and any additional external disturbance. The controller modifies its target position according to the difference among these quantities. As the environment’s reaction approaches the requested force, the extra displacement disappears and the system reaches an equilibrium. Different command settings recover several familiar behaviors:. With no force command and no disturbance, the target is simply the commanded end-effector position. With a nonzero force command, the robot advances until the estimated reaction force balances that command. With a position command and an external disturbance, the robot yields proportionally rather than rigidly rejecting the disturbance. With position control along a surface and force control normal to it, the same policy implements hybrid control for tasks such as wiping. With a zero-force target and a dynamically updated position reference, the end effector follows an applied push and remains at its new location after the push disappears. This is a command-level unification rather than a discrete mode switch. The network does not select among four separately trained controllers. It receives a combination of position, base-velocity, end-effector-force, and base-force commands, and the combination determines the behavior. The robot base receives an analogous treatment, but in velocity space. Because global base position is not assumed to be available, an estimated external force changes the target base velocity instead of a target base pose. This produces behaviors such as walking in response to a push or leaning to resist a commanded load. End-effector and base force compensation are modeled independently, however; UniFP does not yet solve a coupled multi-contact wrench-allocation problem. One subtlety is that the policy is not identifying contact force in a strict metrological sense. It estimates a net external force containing both environmental reaction and disturbance. That is enough for closed-loop compensation, because the active force command is already known. But the estimate does not explain where the force originated, what contact produced it, or how it should be distributed among multiple contacts. UniFP is closer to a learned disturbance observer than to a virtual six-axis force/torque sensor. A learned disturbance observer inside a whole-body policy. The policy operates from proprioception and command history. Its observations include body orientation, base angular velocity, joint positions and velocities, the previous action, task commands, and gait phase information. A 32-step history is encoded into a latent feature that is consumed by both the actor and an auxiliary state-estimation head. That head predicts base linear velocity, world-aligned end-effector position, end-effector force, and base force. The actor ultimately outputs residual joint-position targets, which are added to a default pose and tracked by low-level PD control. The released implementation makes this structure especially concrete. For the B2-Z1 configuration, each observation frame has 73 values, so the 32-frame policy history contains 2,336 inputs. An encoder maps that history through hidden layers of 512, 256, and 128 units to a 64-dimensional latent. A decoder maps the latent to 12 predicted values: three each for base velocity, end-effector position, end-effector force, and base force. The actor combines the latent with the current 73-dimensional observation and produces 17 residual joint commands. A privileged critic is trained from a separate stacked state representation. Notably, the actor receives the latent representation rather than the decoded 12-dimensional estimate. The auxiliary prediction loss encourages the latent to contain physically meaningful information, but does not force the controller to throw away other useful historical cues. The decoded force can then be exported for logging, teleoperation, and high-level imitation learning. This design uses privileged simulation information in a familiar but effective way. During training, the simulator provides ground-truth external forces, end-effector location, and base velocity. The network learns to reconstruct those quantities from deployable observations. At run time, the privileged labels disappear, but the observation-history encoder remains. Because forces change how the robot accelerates, how joints deflect under PD control, and how the base tilts, a temporal model can infer them indirectly. That inference is only possible if contact produces an observable response. The authors report an interesting control-estimation trade-off: lower position gains permit more displacement and improve force estimation, but overly low gains degrade position tracking. In other words, observability is being co-designed with control. An infinitely stiff servo would hide the kinematic evidence needed by a sensorless force observer; a very soft servo would provide evidence but fail to track accurately. The end-effector estimate is also more consequential than it first appears. Straightforward forward kinematics gives the arm pose relative to the arm base. UniFP instead predicts the end-effector position in a world-aligned frame tied to the projected robot base. Near the edge of the arm workspace, a downward command can therefore induce the quadruped to lean or reposition its body, extending reach without an explicit planner deciding how to divide motion between arm and legs. Training contact behavior without contact tasks. Training uses PPO in Isaac Gym with 4,096 parallel environments. The policy first learns whole-body reaching and locomotion. Random force commands and external disturbances are introduced only in a second stage, because adding them from the beginning makes it difficult for an immature policy to learn basic balance and end-effector stability. The simulator samples end-effector positions, base velocities, force commands, and externally applied forces. End-effector force commands and disturbances span up to roughly 60 newtons in either direction. Forces are ramped up, held, and ramped down rather than appearing as instantaneous steps. The reaction force of a particular object is not explicitly modeled in the training formulation; reaction and disturbance are folded into a single net external-force signal. This is both elegant and limiting. It lets the team cover a broad command space without building thousands of object-specific contact tasks. The policy learns the more general skill of compensating a measured disturbance. But random applied forces do not capture every property of real contact: impacts, stick-slip transitions, contact-normal changes, friction cones, object deformation, or coupled moments. The bet is that learning the disturbance-response relationship transfers more broadly than learning each object interaction separately. Domain randomization covers friction, body mass, center of mass, motor strength, gripper payload, and periodic robot pushes. Rewards combine end-effector and base command tracking with penalties on collisions, torque, joint speed, acceleration, action rate, and joint-limit violations. Turning the estimator into a force-aware Diffusion Policy. The second half of the paper is what makes UniFP more than a low-level controller study. The authors use the learned controller as the foundation for teleoperation. Two RealSense cameras—one near the arm and one on the body—record visual observations alongside robot state, operator commands, and estimated contact force. A custom wireless-controller interface commands base motion and end-effector position; holding a shoulder button changes the end-effector input from position commands to force commands. These demonstrations train a force-aware Diffusion Policy. Diffusion Policy is a conditional generative model over action sequences, particularly useful when demonstrations contain multimodal or temporally correlated behavior. In the original formulation, it denoises a sequence of candidate actions and executes them in a receding-horizon loop. UniFP does not fundamentally alter that generative machinery. Instead, it changes the policy’s sensory and action vocabulary. The high-level model receives images, robot state, and the low-level policy’s force estimates. It predicts both end-effector position and force commands. UniFP then converts those task-space commands into coordinated leg-and-arm motion. This creates a clean hierarchy:. The diffusion model decides where to move and how hard to press. The low-level policy decides how the entire legged body should move to make that happen. This hierarchy also addresses a data problem. Most trajectory demonstrations describe what motion occurred, but not what contact made the motion successful. UniFP effectively annotates demonstrations with a learned force channel, even though the robot carries no dedicated force sensor. The estimator is therefore doing double duty: it closes the low-level control loop and supplies contact state to the high-level learner. For wiping, the team collected 50 trajectories and trained for 30,000 steps. For opening and closing a cabinet and opening an occluded drawer, it collected 30 episodes per task and trained for 20,000 steps. The authors note that their button-based teleoperation was relatively slow, taking about 20 seconds for a wiping trial and 10 seconds for a cabinet trial. What the experiments show. The primary quadrupedal platform combines a 12-degree-of-freedom Unitree B2 with a six-degree-of-freedom Z1 arm. The cross-embodiment experiments use a 29-degree-of-freedom Unitree G1 humanoid. Policy inference and Diffusion Policy run on a desktop with an RTX 3090 connected to the B2-Z1 system. In simulation, end-effector position errors remain mostly below 10 centimeters over trajectories spanning the training workspace, while predicted end-effector positions are generally within 5 centimeters of simulator ground truth. On hardware, the researchers command forces from zero to 60 newtons and compare them with a dynamometer at five arm configurations. Average command error stays within 10 newtons, with reported force-estimation errors typically between five and ten newtons. Hardware constraints limit tests on two axes to 40 newtons, and the lateral axis shows the largest sim-to-real discrepancy. Those numbers establish the intended operating regime. This is not precision force metrology for delicate insertion or polishing. It is coarse but actionable force awareness for mobile manipulation, where distinguishing no contact, moderate pressure, and strong resistance may matter more than resolving sub-newton differences. The clearest result is the imitation-learning evaluation:. | Task | Position-only baseline | Force-aware policy | Absolute gain | |---|---:|---:|---:| | Wipe blackboard | 22% | 58% | +36 points | | Open cabinet | 36% | 70% | +34 points | | Close cabinet | 30% | 72% | +42 points | | Open occluded drawer | 30% | 76% | +46 points |. Each entry is measured over 50 physical trials. Averaging the table gives 29.5% success for the baseline and 69% for the force-aware system. Thus, the paper’s headline “39.5% higher” corresponds to 39.5 percentage points, not a 39.5% relative improvement; the relative increase is roughly 134%. The failure modes fit the proposed mechanism. During wiping, a trajectory-only model either loses surface contact or pushes too aggressively. The force-aware model can condition lateral motion on sustained normal pressure. The cabinet mechanism has a narrow three-millimeter activation stroke, making vision alone a poor contact-state sensor. In the drawer experiment, the gripper becomes visually occluded as the interaction progresses. The baseline falls to 30% success, while estimated force provides evidence that the robot has reached and activated the mechanism, raising success to 76%. Beyond imitation learning, the authors demonstrate several manually commanded behaviors. A 25-newton upward command compensates for a 2.5-kilogram dumbbell attached to the end effector. A zero-force command makes the arm follow a human-applied push. Position-plus-force commands produce spring-like behavior during tug-of-war and arm wrestling. Base-force commands let a person physically guide the robot. The humanoid result deserves careful interpretation. The G1 has no manipulator in this experiment, so it receives only locomotion and base-force commands. It can lean and halt in response to an opposing force condition, demonstrating that the formulation transfers to force-conditioned balance and locomotion. It does not yet demonstrate full contact-rich humanoid manipulation. How UniFP relates to neighboring approaches. Deep Whole-Body Control established the value of replacing separately engineered locomotion and manipulation controllers with a single RL policy. Its central concern was coordination: leg motion should help the arm reach, and arm motion should not destabilize locomotion. UniFP inherits that whole-body philosophy but makes force an explicit command and estimated state rather than an implicit by-product of position tracking. The closest learning-based precursor is Learning Force Control for Legged Manipulation by Portela and colleagues. That work also trains a quadrupedal manipulator to regulate force without force sensing, demonstrating gravity compensation, variable compliance, and intuitive whole-body teleoperation. UniFP’s distinguishing move is to expose force and position simultaneously through a unified command interface and to reuse the estimator as a contact channel for imitation learning. The overlap is substantial, so the “first unified policy” claim depends on the precise meaning assigned to unified command-conditioned control. Adaptive Compliance Policy approaches contact from the high-level imitation side. It learns spatially and temporally varying approximate compliance profiles from demonstrations, allowing a diffusion-guided controller to avoid excessive contact forces while retaining task accuracy. That is more expressive than UniFP’s simplified, largely fixed impedance relation. UniFP, however, addresses a harder whole-body deployment problem and attempts to infer force using ordinary legged-robot proprioception. Diffusion Policy itself is not a force-control method. It provides the sequence-modeling backbone, including multimodal action generation and receding-horizon execution. UniFP’s result suggests that changing the action interface and observation schema may be as important as changing the generative architecture. A powerful visuomotor model cannot recover contact information that is absent from both its demonstrations and its observations. More recent work is also exploring the opposite engineering trade-off: making explicit force sensing cheaper and easier to collect. UMI-FT, for example, uses compact six-axis sensors in a handheld demonstration device and learns policies that predict position, grasp force, and stiffness. Sensorless estimation offers lower hardware complexity, while explicit sensing offers better observability and ground truth. The likely long-term systems will combine both, using learned estimation for robustness and physical sensing where precision or safety requires it. Why this paper matters. The most transferable idea in UniFP is its policy interface. High-level robot learning commonly assumes that the useful action is a desired pose or joint trajectory. UniFP argues that the action should sometimes be a pose-force pair. That changes what demonstrations must contain, what a visuomotor model predicts, and what a low-level controller promises to execute. The second strong idea is treating an auxiliary physical prediction as reusable infrastructure. The force estimator is not merely a regularizer discarded after RL training. Its predictions become observations for the task policy and annotations for demonstrations. This creates a bridge between simulation-only privileged information and real-world data collection. Finally, the method separates two kinds of generalization. The low-level policy generalizes across combinations of position, force, disturbance, and body posture. The high-level Diffusion Policy generalizes across task observations and demonstration trajectories. Neither model must independently learn the entire mapping from pixels to whole-body joint torques. Where the evidence is still incomplete. The first limitation is fundamental observability. A history of joint motion can reveal that something is resisting the robot, but it cannot always determine why. Payload error, actuator friction, terrain contact, calibration drift, external pushes, and end-effector contact can produce similar proprioceptive signatures. UniFP estimates a three-dimensional net force, not a localized six-dimensional wrench with a known contact point. This matters for multi-contact behavior. The current formulation treats base and end effector separately and concentrates on one principal interaction point. It does not decide how a quadruped should distribute forces among its arm, body, and feet while bracing against an object. The authors explicitly identify coordinated multi-point force estimation and interaction as future work. The second limitation is the quasi-static approximation. Omitting end-effector inertia and damping is reasonable for slow wiping, pushing, and human-guided interaction. It is much less convincing for hammering, catching, impact-rich tool use, fast door opening, or contacts whose normal changes rapidly. The paper reports degraded estimation during high-frequency interactions and near workspace boundaries. Third, synthetic applied forces are not a replacement for diverse simulated contact. Random load injection teaches disturbance rejection and compliance, but does not directly teach contact mode transitions, frictional constraints, impacts, or object dynamics. A stronger successor could combine UniFP’s broad force randomization with procedurally generated articulated objects and randomized material properties. Fourth, the real-world force errors are large for delicate manipulation. Five to ten newtons may be acceptable for pressing cabinet latches or supporting a payload, but it would be risky for plugging connectors, handling fragile objects, or controlling a cutting tool. Temporary force/torque instrumentation would still be valuable for calibration, validation, and safety even if the final deployed robot runs sensorless. The imitation comparison is also a system-level ablation rather than a clean attribution study. The baseline removes both force estimates and force-command signals during demonstration collection. The experiment therefore establishes that the complete force-aware interface is better than a position-only interface, but it does not isolate how much improvement comes from contact observation, force-conditioned actions, compliant low-level control, or altered demonstrations. There is no extensive comparison against the preceding learned legged force controller, an adaptive impedance baseline, or a well-tuned model-based hybrid controller. Likewise, cross-embodiment evidence consists of quadrupedal manipulation and humanoid base-force behavior, not the same manipulation suite transferred between embodiments. Finally, reproducibility is still partial. As of August 5, 2026, the public repository contains the B2-Z1 Isaac Gym environment, PPO implementation, robot assets, and training commands. Its README nevertheless continues to mark the complete training release, ROS 2 sim-to-real pipeline, MuJoCo sim-to-sim support, and imitation-learning data-collection pipeline as unfinished items. How I would adapt UniFP. For a team building a contact-rich legged manipulation system, the most practical first step would be to adopt the interface rather than copy the complete implementation. Define a task-space command containing base motion, end-effector pose, and desired force. Train a low-level controller to cover the Cartesian product of those commands, including combinations that appear unusual but reveal whether the policy has genuinely learned compositional behavior. Next, treat force estimation as privileged representation learning. Apply known forces at multiple body locations in simulation, supervise per-contact estimates, and ask whether the latent distinguishes payload, base disturbance, foot contact, and manipulator contact. For humanoids or robots expected to brace with their bodies, a set- or graph-based representation of candidate contacts would be more appropriate than one end-effector vector and one base vector. Real-system calibration should use an instrumented test fixture even if production hardware will omit the sensor. Measure estimator bias across workspace position, payload, motor temperature, velocity, and contact stiffness. Fine-tuning on real force labels or fitting a small residual estimator would directly address the sim-to-real gap identified in the paper. For high-level learning, demonstrations should store both the operator’s intended force command and the low-level estimate of achieved force. Those are different quantities, and their difference tells the high-level model whether the intended contact has been established. A sequence model can then predict pose, force, and possibly stiffness or damping, while the low-level layer enforces feasibility. Safety should not be delegated entirely to the learned policy. Force-command saturation, torque limits, collision detection, workspace constraints, and passivity- or energy-based monitoring are natural additions. The need is especially acute when force estimates are uncertain: a controller that mistakes servo friction for missing contact may continue pushing when it should stop. The larger lesson. UniFP does not replace classical force control so much as repackage its essential structure for modern robot learning. It turns force from a hidden consequence of motion into part of the policy’s command language, latent state, and demonstration data. That shift is why the work is important. The reported force estimates are coarse, the contact model is simplified, and the humanoid evaluation is preliminary. Yet the paper demonstrates a credible route from privileged simulated forces to sensorless real-world compliance—and then from low-level compliance to better visuomotor imitation. For embodied AI, the broader message is simple: a trajectory is not always the action, and an image is not always the observation. In contact-rich robotics, the missing variables are often the forces connecting the robot to the world.