Latest / Embodied AI 101 / Learning Unified Force and Position Control for Legged Loco-Manipulation

Transcript

Learning Unified Force and Position Control for Legged Loco-Manipulation · Embodied AI 101

• Learning Unified Force and Position Control for Legged Loco-Manipulation. Legged robots – especially quadrupeds and humanoids – are increasingly expected to do more than just walk. They need to manipulate the world around them: pushing open doors, carrying heavy objects, bracing against walls, or any task that requires carefully coordinated use of the feet, legs, and/or arms. This combined task of walking or balancing and interacting physically with the environment is often called loco-manipulation. It’s a prime example of “embodied AI” in action, because the robot’s intelligence must include control of its body in physical contact with the world. Crucially, many loco-manipulation tasks need not only positioning but also deliberate application of forces: think of a quadruped robot pushing a stuck crate or a humanoid pressing its arm against a heavy valve. Successful execution requires controlling not just where the limbs move (position control) but also how hard they push or hold (force control). Legged manipulators have an advantage for these tasks. A quadruped with an arm (for example) or a humanoid can leverage its whole body. They can combine terrain adaptation and extended reach. For instance, a four-legged robot might bend its front legs to place its end-effector on the ground, or raise up its body to reach a high shelf. This combination of mobility and manipulation massively expands what the robot can do. However, it also creates a complex control problem. The robot must simultaneously keep itself stable (balancing, stepping over rough terrain, etc.) and apply desired forces at contact points. Traditionally, engineers would break this problem into parts: one controller for locomotion, another for manipulation or contact forces, and then an external planner to coordinate them. But coordinating separate controllers can be brittle, especially when the environment or the contact target is uncertain. The new work Learning Unified Force and Position Control for Legged Loco-Manipulation proposes a different approach. Instead of separate modules, the authors train a single reinforcement-learning (RL) policy that handles everything together. In this unified framework, the policy learns to control the robot’s joints to achieve both position goals (like moving the foot to a target location) and force goals (like pushing against an object with a specified force), all in one learned behavior. Remarkably, this policy is trained without any force sensors on the robot. The policy never directly measures contact forces; instead it learns to infer and control forces indirectly through the usual proprioceptive feedback (joint encoders, IMU, etc.) and careful reward design. As a result, the policy can seamlessly switch between free motion and contact tasks, track desired contact forces, and exhibit compliant (springy, protective) behaviors during interactions – all while under a single neural-network policy. In this episode we’ll unpack how they did this, why it matters for robot control, and what kinds of tasks it enables. We’ll pay special attention to what “force-aware imitation learning” means, and how a robot without a force-torque sensor can nonetheless execute gentle push-and-pull tasks. Along the way, we’ll tie the ideas into the broader context of whole-body control research – for example, recent work has even demonstrated learned policies that explicitly target both position and force simultaneously. The novelty here is that the new policy achieves similar capabilities from learning alone, no explicit force feedback needed. Let’s dive in. Position Control vs. Force Control. To appreciate the challenge, let’s recall what position control and force control mean in robotics. In position control, the robot is given a desired trajectory or goal pose and it uses its actuators to follow that path. For example, you might command a robot’s hand (or foot) to move to a particular spot in space. The controller then would ensure the robot’s joints move so that the end-effector reaches that target. This works well when there is no external contact or when contacts occur but the goal is purely positional (for example, setting the foot down exactly at a spot). Position control is like saying, “Move to here,” and trusting that the environment will simply be where it expects. In force control, however, the goal is different: you tell the robot how hard to push or press on something, rather than an exact position to go to. For instance, you might want the robot to push a heavy box with a constant 50 N of force, or to gently press a button requiring 10 N. Achieving a target force typically requires the robot to adjust its pose on-the-fly – if it presses too hard, it might back off a bit; too little, it might move in more. In principle, force control is often handled with special controllers (like impedance/admittance control) that modulate motor torques or joint compliance based on error in force. But crucially, tracking a force target normally assumes you can measure the contact force (with a force-torque sensor or tactile sensor). Long-standing robotics research on hybrid position/force control (going back to the 1980s) recognizes that many tasks involve both objectives: some directions of motion are position-controlled while others are force-controlled. For example, when a robot pushes along a wall, you might fix the robot’s orientation but only specify a force normal to the wall. These controllers often transform the problem into task space (where the force and motion constraints apply) and solve a control law that mixes position commands and force feedback. Even without learning, whole-body controllers can include force tasks (for instance, specifying foot-ground contact forces). But these classic approaches typically rely on having a model of the robot’s dynamics and actual force measurements. The key insight here is that an RL policy doesn’t need explicit models of dynamics or sensors of everything. Instead, it can learn from trial and error to produce behaviors that implicitly achieve force control goals. Think of it this way: a reinforcement learner sees the result of its actions through whatever sensors it has – joint encoders, gyros, maybe depth cameras, etc. It doesn’t have a direct “force” readout. If the desired outcome is to push an object with a certain pressure, the environment (simulation) can still compute how much force resulted, and that can be fed into the reward. The policy then indirectly learns how much actuator effort or pose adjustment is needed to hit that force target, even if it never “sees” the force. As the authors put it, by training in simulation they can shape rewards for both position tracking and force tracking. The policy’s neural network might output joint torque commands or some other control signals, and the environment will compute what force those produce at the contact point. If the policy learns that reducing its step angle or stiffening a joint increases the push force, it can infer how to achieve the force objective. In effect, the robot “guesses” how hard it must push (by controlling motor torques) and is rewarded when the actual contact forces match the target, despite having no explicit force sensor in the loop. Why do this? Force control is essential for compliant interactions. If the robot has no sense of force, it might slam into obstacles, or fail to apply enough pressure. By integrating force objectives into the learning, the policy naturally becomes compliant. For example, if the target force is smaller than the robot’s current push, the policy back-offs to avoid excess force; if the robot is bumped by an external load, it can relax or counteract accordingly. Compliance here means the robot behaves softly around unknown contacts. In position-only policies, any unexpected contact might be treated as an obstruction and cause erratic responses. With force objectives, the robot tolerates and adapts to forces. In summary, position control tells a robot where to move; force control tells it how hard to press. The new RL policy learns both at once. We’ll see below how the authors set up the learning tasks so that the same policy controls limb positions and also satisfies force targets whenever needed. The Unified RL Policy: States, Actions, and Objectives. The heart of the approach is to design an RL problem where both position and force goals are present. Concretely, the policy (a neural network) receives observations that include the robot’s joint angles, velocities, base orientation, etc. Possibly it also sees task-specific information, like the desired foot placement or end-effector location. It does not receive any direct force measurement. The robot class could be a simulated Anymal quadruped or a Humanoid, depending on the scenario. What does the policy output? Typically in these loco-manipulation tasks, the network issues commands in joint space (like target torques, joint velocities, or even angles for a low-level PD controller). It might output desired joint positions with some learned stiffness, or just raw torques if that’s stable enough. The crucial part is that the action space and the reward design allow for force control behaviors to emerge. For a task requiring force control, the reward might include a term that compares the actual contact force to the desired contact force. For instance, if a robot’s hand is supposed to press with 20 N on a door panel, the simulator can compute the instantaneous contact force on that panel, and a reward penalty can be the square error between actual and 20 N. If the robot does exactly 20 N of push, it gets max reward for that term. If it pushes harder or softer, it is penalized. Other terms in the reward could be position tracking errors (how far the end effector is from a target location), plus generic terms for stability, energy usage, etc. Importantly, while the reward depends on forces, the policy input does not. The policy gets standard proprioception. This means the learning algorithm must figure out, from correlations, what actions cause the right amount of force. Over many training episodes, it will discover strategies: for example, if the robot‘s foot is fully extended and stiff, it will measure something (like motor torques) indicating a high force and then adjust. If the foot is not touching, it will learn to advance it until contact, then regulate. Essentially, it “learns to feel force through forceful action” without feeling. It appears the authors implemented a multi-task reward solution: some tasks are purely positional (walk to a location, reach a point) and some are force-oriented (apply pressure to a wall or object). By training on a variety of tasks, the single policy can learn both modalities. One episode might demand tracking a moving target (position control), while another might have a fixed target plus a specified force. The unified policy uses the same network for both. Over training, it builds an internal understanding that certain actions yield motion, others yield force, and that it needs to coordinate. Since we lack direct force feedback, the policy might rely on other signals. A plausible hint is the joint torque or acceleration: if you command a joint towards a target position but observe it doesn’t move (because it’s pushing against something), the controller senses internal torque rise. In many actuators this is measurable as needed effort. The policy can get joint effort as part of its input. The summary doesn’t explicitly say, but often in such RL setups joint torques or motor currents are included in the observation. These serve as an indirect force sensor. The network could learn that “if torque climbs above some level while the joint hasn’t achieved its motion target, I am in contact and must adjust.” Without a force sensor, torque is the next best proxy. Another design trick can be variable impedance: in classical control, you might specify a stiff or soft joint depending on task (e.g. compliance mode vs tracking mode). Perhaps here, the policy’s outputs effectively set the robot’s limb stiffness implicitly. For instance, if it outputs high PD gains, the joint will resist deviation strongly (and produce high contact force). If it outputs low gain, it will give way easily (softly touch). The network could learn to modulate how “stiff” to act based on whether it needs force or position tracking. This is speculation, but one way to see “compliance” emerge from policy control might be that in feely momentum moves it sets itself stiff for precision, but when contacting it lowers gains to not slam. In any case, the unified policy is trained end-to-end with something like PPO or SAC. The learning environment would simulate the legged robot and let it try actions, computing both position error and force error rewards. Over time, the policy learns a single strategy that satisfies both demands. Because it’s all one learning problem, the policy automatically learns to trade off between pushing and moving. (For example, if it tries to hold a pose while pushing, it might slow down its gait or lean into a wall.) In essence, position goals and force goals become parts of one overall objective. A bonus: this unified approach also naturally handles transitions between free motion and contact. In a hand-crafted controller you might need to detect a contact event and switch control modes. Here the policy simply faces different rewards: if it’s told to track a force, it will gently press; if no force is needed, it just moves freely. If an unintended bump happens, the learned compliance helps. Position and Force Tracking Tasks. To see how this works, consider a concrete example task: a quadruped robot must push open a heavy cart that’s initially fixed but can slide with enough force. The goal could be two-fold: move one contact limb (say, a front leg or an arm) to a goal patch on the cart, and apply a constant 50 N of force while doing so. The reward might include (1) a bonus for placing the end-effector at the right spot on the cart and (2) a penalty proportional to the square of the difference between actual push force and 50 N. Additionally, the robot has to keep walking or balance. Initially, before contact, the policy focuses on moving the limb to the target location (position task). Once contact happens, the policy now cares more about force: if it detects that pushing is happening (through increased joint torques or the fact it’s not moving), it will adjust its action to make sure the push stays at 50 N. If it overshoots (say it’s pushing at 60 N), it might ease off by slightly retreating or relaxing the joint. If it’s only pushing 30 N, maybe it leans harder or braces differently. Because the reward encompasses both objectives simultaneously, the learned policy effectively tracks both a trajectory and a force profile. The authors describe this as “position+force tracking.” That means in a single task run, they measure both how well the robot follows a position goal and how well it maintains force. It’s something normally hard to do with one controller. This capability of tracking force and position with one policy is powerful. It allows sophisticated behaviors. For example, a humanoid could walk up to a wall (position) and then exactly press a hand against it with a particular force, sliding along if needed, all in one routine. Or a quadruped could step on a tilt pedal and apply enough force to hold it down while continuing to move in place. Normally, implementing either of those tasks would need careful switching between gait generation and contact regulation. Here it’s just one neural controller that learned it. The policy being trained without force sensors would still be evaluated on actual force output. In simulation, the ground-truth contact force (from the physics engine) is known. So they can compute the “force error” even if the policy doesn’t know it directly. In deployment on a real robot, they wouldn’t have that term in loop. But one could use estimation or even add a cheap force sensor. The key is that training shaped the policy’s behavior. When the authors mention “position+force tracking,” they highlight that the robot can concurrently keep a body posture or limb pose (the position part) and meet a force requirement. For locomotion this might look like walking while maintaining pressure on a support, similar to a person deliberately leaning against moving belt at a constant push. The policy causes the robot to tread with a specific weight distribution and limb stiffness. Another nuance is “whole-body coordination”. If, for example, a humanoid pushes an object with its hand, the force it exerts will cause reaction forces through the body. The robot must adjust leg forces to remain stable. A unified policy inherently coordinates that: it might shift its center of mass or reorient feet to counterbalance. Because the learning reward can also include stability metrics (e.g., keeping balance or minimizing joint torques for stability), the policy learns to spread forces across limbs. This way the robot doesn’t tip over while pushing. Overall, the training involves many different scenarios of contact. Some might be purely position tasks (like reaching in free space), others purely force (like pushing in place), and many in between. The variety ensures the policy encounters when to emphasize position vs. force. By the end, it can fluidly blend. In practice, the robot is given instructions in a higher-level sense (like “follow this path and if you hit an object apply force F”), and the learned policy carries it out. Importantly, all of this happens without any force sensor input to the policy. The network is essentially blind to actual contact magnitudes, and must reason indirectly. This is similar to how humans can learn to push gently without a force gauge – we just feel it in our muscles and adjust. The robot’s “muscle feel” comes from proprioceptive cues and the learned model. Emergent Compliance and Safety. A striking result of this unified learning setup is that compliance – a kind of soft, yielding behavior in contacts – emerges naturally. Because the policy is optimizing for appropriate force, one outcome is that it will tolerate a range of forces without over-reacting. For example, if an external object bumps into the robot’s leg unexpectedly, a purely position-focused controller might try to maintain its trajectory no matter what (leading to a stiff response or even damage). Here, the policy learned that limits of force are targets, not walls, so if a bump occurs, it can yield slightly and re-apply contact gently. That is the robot being compliant. Compliance is also crucial for following the environment. Imagine a quadruped walking and lightly scanning its way through a cluttered area. If it bumps a wall with a leg, a compliant robot will simply slide along, whereas a rigid one might get stuck or fall. The unified policy seems to acquire that gentleness because doing so often leads to better overall rewards: if the robot, say, is pushing an object, easily adjusting to any change in object reaction is key for tracking force. If the object shifts or gives a bit, the robot must adjust – which is a form of compliance. This approach also helps with safety. During training, the policy may explore many contact scenarios. It learns to avoid excessive forces because the reward punishes force error. In a real setting, this means the learned policy inherently has a notion of not overshooting forces. In other words, the robot learns to be gentle, to the extent that the task allows. For humans interacting with such a robot, that means fewer surprising knocks. In summary, emergent compliance is a byproduct of crafting rewards around force targets. When the reward encourages matching a moderate force, the easiest solution is often to be softly pushing rather than rigid. The policy thus values soft behavior. This is beneficial in applications like handling delicate objects or operating near people. A future robot with such a policy might not slam into obstacles, making it more suitable for unstructured spaces. Force-Aware Imitation Learning. One intriguing element mentioned in the summary is “force-aware imitation learning.” Let’s break down what this likely means. Imitation learning means using demonstration data (for example, from a human or from an optimal controller) to teach the policy an initial skill before or during RL. If the environment is “contact-rich,” demonstrations might include examples of making contact with particular forces. Being "force-aware" suggests the imitation not only copies motions but also replicates force outcomes. Here’s a plausible scenario: The authors might have a dataset of demonstration trajectories for tasks like pushing or pressing, where the demonstrations also provide what the force at contact should be (maybe the demonstrator was a scripted controller that knows the physics). The policy could be pretrained or guided to match both the demonstrator’s joint trajectories and the implicitly associated forces. For example, one could add a term to the imitation loss that rewards producing the same contact force as the demo, even though the policy has no direct force input. This trains the policy to guess how to achieve the demonstrated force profile. Another angle is that “force-aware imitation” means designing the imitation tasks so that they include environmental contacts explicitly. Many imitation approaches only consider proprioceptive imitation error (difference in joint angles/velocities). But here, one could include the environment outcome in the imitation objectives. For instance, even if a robot’s hand is too high, if pressure is too low compared to a demonstration, that would need correction. Concretely, one might alternate between RL and behavior cloning steps. A special loss could evaluate the difference between the robot’s end-effector force and the demo’s end-effector force when at the same pose. Even though the policy doesn’t feel the force, the simulator can measure it and provide a target for the imitation loss. Essentially, the policy tries to produce actions that would yield the demonstrator’s contact forces. This enriched imitation might accelerate learning. Instead of random exploration having to discover, say, “to push open door, use 30 N,” you give it a strong hint: the demo says “this is roughly how much force you need here.” The policy then only has to fine-tune from a good initialization. In contact-rich tasks, UE errors can be large, so imitation is helpful. It’s also possible they incorporate imitation in the reward: like “if your current contact force is far from the demonstration’s force, punish it.” That would be an imitation penalty focusing on force. The overall phrase “force-aware imitation” suggests the demonstrations or imitation objective incorporate force tracking explicitly, not just motion mimicry. What tasks would benefit? Imagine a humanoid that’s supposed to slide crates or open doors, guided by a teleoperated demo. If the human was demonstrating with knowledge of how much push is needed, then the demo includes force data. Alternatively, synthetic demos (like planning in simulation) could inject force guidelines. The summary doesn’t give full detail, but presumably they found that including this imitation step improved performance on tasks where pure RL was too slow. Particularly, it helps for tasks where the exact contact profile is tricky to discover by random tries. The result is a policy that not only controls position but has an embedded understanding of how to shape forces, thanks to the imitation guidance. It’s like giving the robot a tutor for physical contact tasks. Whether imitation learning is used as a pretraining or as a continual penalty term, the effect is to make the RL policy “force-aware” from the very beginning. They can then fine-tune with RL’s exploration to polish the skill and generalize to variations. In practice, the paper reports kinds of tasks called “contact-rich”. These might include things like rubbing an object against a surface, sliding along while pushing, or tracking a force pattern over a complex motion. The policy trained with force-aware imitation would learn to match the intended force profile in those cast. Without imitation, an RL attempt might wander aimlessly; with imitation, it has a head start. The big picture: combining force-aware imitation with the unified RL approach means the robot leverages demonstration knowledge where available, then uses trial-and-error to refine and handle randomness. This hybrid strategy often leads to more efficient learning for complex tasks. In a field now exploring scaling up from point A to B, adding forces is another dimension – and imitation helps tackle that high-dimensional target. Experiments: Quadruped and Humanoid Tasks. What did this look like in practice? While I don’t have the exact tasks from the paper (the detailed results aren’t in our text box), we can imagine the kind of challenges they must have used to validate this approach, and draw on hints in the summary. The key words are “quadrupedal and humanoid” robots, “contact-rich tasks,” and capabilities like position+force tracking and compliance. For a quadrupedal robot with an arm (think: ANYmal with an attached manipulator, or a four-leg robot that uses one leg as a pushing limb), tasks might include:. Pushing a Box or Cart: The robot approaches a box and must push it straight, requiring a constant forward force while stepping. The task is successful if the box moves a certain distance without tipping and the robot doesn’t topple. The policy must keep walking in place or stepping while exerting pressure. Here it must track both foot positions (to stay balanced) and hand/leg force on the box. Pulling or Carrying: Though pulling is slightly different (tensile force), a similar concept: the robot carries a payload via a strap and must regulate the traction force. The task is to move while keeping the carried object at a safe pull tension. Leaning or Stiff Leg: Maybe a scenario where the robot has one leg pinned to a spot for support (e.g., to hold up a beam) with a target support force. It learns to lean against something with a specified force to stabilize an object overhead. This tests ability to hold posture and force. Walking on Uneven Ground with Force Constraints: For instance, suppose one leg steps on a slippery tile. The robot might have a target foot pressure to maintain – too little and it slips, too much and it might break something. The unified controller would adjust posture to hit a target thrust. Manipulating Valves or Latches: If a quadruped has a front limb that can twist a valve, tracking a torque (force) target over the twist is required. They could simulate grasping a handle and applying a turning force while standing. For a humanoid robot (like a biped with arms, e.g., a simulated Atlas or TALOS), tasks could be:. Door Opening: The humanoid walks to a door, grasps handle, and applies force-orientation control to pull it open. The force target is along the door’s normal. Meanwhile, it must keep its feet and body stable. This requires moving (locomotion towards door) and then switching/combining to pushing/pulling force. Carrying or Pressing Button: Walking plus holding an object of specified weight, or pressing a large button with exact pressure to not break it. The hand force must match a target, while balancing on legs. Climbing Ladder or Stairs: If climbing, one might apply a constant force on a rung with one hand while stepping with feet. The force ensures grip, the stepping is position. Postural Support: For example, lean against a wall with arms to extend a reach or to brace while doing something else. The target might be normal force on wall, with arms, while feet hold balance. Assembly Tasks: Perhaps a bit advanced, but e.g., the humanoid aligns a heavy mass and uses both arms to press a component together (force in one direction, position in others). Each of these tasks has a mixture of moving the limbs to the right spots and controlling how strongly they interact. The unified RL policy would be trained on a suite of such tasks or combinations thereof, hopefully with randomization in targets to encourage generality. In the experiments, the authors likely compare this unified policy against baselines such as:. A standard locomotion policy that ignores forces (so it might walk but slam into contacts). A separate force-only policy (for example, one that is great at static push but can’t move). A hybrid approach (maybe switching between a force controller and a position controller, but not learned). They would measure metrics like success rate (did the box reach destination?), tracking error (how well did the force match the target?), and robustness (did it stay up when pushed unexpectedly?). For example, say a humanoid is trained to push a heavy object requiring 100 N force. The unified RL policy should get the object moving and keep it moving steadily. A position-only policy might move until contact and then get stuck (giving an impulse but not pushing further). A force-only policy (if it existed) might push in place but not reposition its feet properly. The unified policy could learn to adjust the legs to lean into the push as needed – achieving both tasks. Another likely set of experiments: compliance tests. For instance, they may drop an unexpected load on the robot’s arm or leg and see if the policy holds without breaking posture. A compliant policy would yield slightly, absorbing impact, whereas a non-compliant one might jerk or fall. The reward shaping for force would encourage gentle yielding rather than stiff resistance. The summary’s mention of "force-aware imitation learning for contact-rich tasks" suggests one scenario: perhaps they collected (or generated) demonstration clips of these tasks and incorporated them. An experiment could compare learning with imitation vs without. The force-aware version probably learns faster or avoids failures. Finally, because they say the policy works for both quadrupeds and humanoids, they likely train separate policies for each robot (since the dynamics differ) or a single architecture applied in two families of tasks. They’d show that the same learning framework can handle both 4-legged and 2-legged cases. That indicates generality (the policy representation is likely robot-agnostic aside from input size differences, or they use two networks). All these results would demonstrate that the unified policy indeed enables a robot to do complex loco-manipulation that involves delicate force control. If they have videos (as many of these works do on project pages), we might see a quadruped soldiering on while its arm pushes a door open, or a humanoid tapping a surface with set pressure. Importantly, they claim zero explicit force sensors. That likely means any real-hardware tests (if any) would use only proprioception. It might also mean their approach is ready for robots that, say, are full-on manipulators but have cheap leg links (no force-torque sensors at ankles or wrists). It’s a practical advantage: cheaper, simpler robots can still do forceful tasks if their controllers are trained cleverly. Putting It in Context. This work joins a growing line of research toward unified whole-body control via learning. Traditional control approaches (MPC, QP-based whole-body controllers) have been extended recently with RL to improve robustness and adaptivity. For example, “Deep whole-body control” and “Unified Mobile Manipulator Learning” show how a legged robot can learn locomotion plus arm manipulation. The new twist here is explicitly focusing on force control within the learning framework, without sensors. Indeed, researchers have started to tackle force control in legged contexts. One recent CoRL paper, for example, explicitly trains a policy for both position and force control in legged loco-manipulation. Another ICRA work looked at learning force control for a legged manipulator. The difference is subtle but important: those works generally presume the controller can access force information (or they structure tasks with known forces). By contrast, the current approach shows that even without force feedback, an RL agent can nonetheless acquire forceful skills. In practical terms, that means robots without expensive force-torque sensors (like an F/T at each foot or wrist) might still do things like yield gently or push with care. The system learns to interpret proprioceptive cues instead. In a sense, it’s a form of sensor fusion where the network learns an implicit model of contact dynamics from the proprioceptive signals. Importantly, this also spreads out the robotic whole-body control story: unified RL avoids some limitations of classical controllers. Model-based methods need accurate dynamics and contact models; if a real-world contact is slightly different, they falter. A learned policy, trained with noise and variety in simulation, can be more robust. Even handling entire families of robots (quadruped vs humanoid) might be feasible with the same learning architecture, while parameterized for each morphology. For researchers in RL and robotics, this work bridges a gap: it says “Yes, you can integrate force tasks into end-to-end learning without adding specialized hardware.” That could inspire others to look at purely vision-based force estimation, or more clever proxies, combined with learning, to get forceful skills. It also ties into trends of loco-manipulation. There’s been a flurry of interest in making legged robots do manipulation (multi-locomotion tasks). Papers like TAC-LOCO used tactile sensors on grippers, RoboDuet taught coordinated arm+leg tasks, and others focus on imitation or diffusion models for whole-body skills. The unified position+force policy is another tool in that toolkit, and it’s especially relevant for environments where sensing is minimal. Finally, this approach suggests new benchmarks. Traditional loco-manip benchmarks emphasize reaching or object relocation without touching anything fragile. We might see new benchmarks where the reward includes matching a contact force profile – like “push this wall at 30 N as you walk by.” Researchers might compare algorithms on “force tracking error” as a metric. Example: Quadruped Bartending (Imaginative Case). To make things concrete in our minds, imagine a task like a robot butler bartending with a quadruped. It walks up to a counter, presses a button to dispense a drink, and catches a glass. Pressing the button might have a delicate spring that needs exactly a 5 N force. The learned policy would carry the quadruped to the panel (position control), then lower an “arm” or leg to press it. As soon as contact happens, the policy switches into force mode: it keeps pushing until the button clicks (tracking that 5 N). A pure position policy might smash the button (exceed force) or not engage fully; a force-aware policy gets it right and then retracts. All the while, the robot’s feet adjust to maintain balance, and if someone bumps into it, it yields smoothly. This kind of nuanced task is exactly what unified position-and-force control enables. Challenges and Future Directions. Training such a unified policy is not trivial. The search space of behaviors is large, and mixing objectives can cause interference. For instance, the policy might learn to over-prioritize either position or force unless balanced carefully. The authors probably solved this by carefully weighting the reward terms and by curriculum learning (gradually increasing force requirements as the robot learns to move). They might have also used domain randomization to ensure robustness (e.g. varying object friction or weight so the policy doesn’t overfit a single scenario). Moreover, sim-to-real transfer is always a concern. The nice property here is that the policy doesn’t rely on perfect sensing, so sensor calibration issues are limited. On the other hand, contact physics in the simulator (friction, compliance) might differ from real life, so the policy might need fine-tuning or domain adaptation. Perhaps future work would deploy it on hardware and assess performance if actual sensors aren’t there. One potential extension: what if we did have a force/torque sensor? The policy architecture could easily incorporate that extra input and possibly perform even better. The current method can be seen as a baseline “no sensor” solution; adding sensors could refine accuracy. Alternatively, vision or tactile skins could provide more contact cues for the policy. On the imitation side, one can imagine using force-aware teleoperation: a human operator with a haptic device could demonstrate tasks, giving perfect demonstrations of both motion and force. That could be fed into the training process to accelerate it. The paper hints they use synthetic demonstrations, but real ones would be an exciting avenue. Finally, while the focus here is on legged robots, the idea of training force control without force sensors could apply to other domains. For example, a manipulator arm with no wrist force sensor but with stiff springs in its joints could similarly learn to control contact forces by monitoring joint deflections. The principle is general: if you can simulate the contact and define a reward, an RL policy can learn to feel its world. Conclusion. In summary, this work presents a new way to give legged robots the nuanced touch they need, using learning-based control. By unifying position and force objectives into a single RL policy – and cleverly circumventing the need for dedicated force measurements – the authors show that contact-heavy tasks become tractable. The robot gains compliant behavior (important for safety and adaptability) and can mimic demonstrations that include force targets. For anyone designing legged robots to work in the messy real world, this suggests a path forward: train your controller not just to move, but to liquefy forces when appropriate and tighten them when needed, all end-to-end. That’s a step closer to truly embodied intelligence.