Latest / Embodied AI 101 / REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering
Transcript
- REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering. Legged robots have achieved remarkable agility in recent years, vaulting over obstacles and performing dynamic parkour maneuvers that once seemed out of reach. However, almost all of these systems rely on near-perfect perception – precise terrain maps, clear camera images, low latency, etc. In practice, even brief visual noise, latency, or sensor dropout can send an agile quadruped reeling. As the authors of this work observe, “recent learning-based systems…remain fundamentally fragile to perceptual degradation, where even brief visual noise or latency can cause catastrophic failure”. Imagine a blind spot or a noisy camera frame, and the robot’s precise foothold placement can fail instantly. The challenge, then, is to build an agile controller that embraces imperfect perception – one that can remember where it has seen obstacles and enforce basic physics consistency, rather than collapse when the vision goes dark for a moment. This paper introduces REAL (Robust Extreme Agility Learning), a comprehensive two-stage framework designed for exactly this challenge. A privileged teacher policy is trained in simulation with full terrain knowledge and learns rich associations between proprioception and ground geometry. Then that policy is distilled into a deployable student that uses only on-board depth sensing and proprioceptive feedback, but with a specialized architecture: a FiLM-based fusion of vision and state features feeding a “Mamba” temporal backbone (a fast state-space network) to build short-term memory of the terrain. Crucially, REAL also adds a physics-guided filter (an EKF built around a learned velocity predictor) to enforce rigid-body consistency during impacts, and an adaptive training “gating” that balances imitation versus RL loss when distilling policy. Combined, these ideas let REAL handle spatio-temporal cues – remembering past observations and filtering noise – to traverse extreme obstacles even if the vision is blinded for a second. In experiments on a Unitree Go2 quadruped, REAL succeeds where others fail: traversing jumps and gaps with a 1-meter-blind zone (no visual depth data in the last meter before the obstacle) while maintaining a tight 13.1 ms inference cost for real-time control. Below we unpack the key ideas. We first describe how the two-stage learning strategy integrates vision, state, and memory (Section 1). We then explain the physics-guided state estimation that keeps the robot consistent under impacts (Section 2) and the consistency-aware training that stabilizes distillation (Section 3). Finally we summarize the experiments showing how REAL outperforms previous parkour-oriented controllers in noisy, blind, and high-impact scenarios (Section 4). Throughout, we connect to related work in legged navigation and high-speed control, framing REAL as the next step toward truly robust and agile parkour. 1. Spatio-Temporal Policy Learning. The heart of REAL is a two-stage training pipeline, illustrated in the paper’s system diagram. In Stage 1, the team trains a privileged teacher policy in simulation using reinforcement learning (RL). This teacher has access to perfect terrain information: the paper mentions “terrain scan points” and additional privileged maps of the environment. It also sees the robot’s full proprioceptive state. With this complete picture, the teacher learns how to map state and terrain to joint commands for extremely agile maneuvers, effectively building a “proprioception–terrain association” model. In practice this might involve things like knowing exactly where upcoming footholds are or how an obstacle geometry relates to the robot’s body orientation, all of which it can use to plan precise leaps or steps. Once the teacher is competent, Stage 2 plays out as distillation into a believable, deployable controller. Instead of perfect terrain, the student policy sees only on-board sensors – for example, a front-facing depth camera and the robot’s own IMU/encoder readings. It must mimic the teacher’s agility without the privileged view. This is fundamentally a cross-modal architecture: the teacher learns by correlating proprioception and terrain, and the student must fuse vision and IMU to recover that information. To equip the student with this multimodal perception, REAL uses a FiLM-modulated Mamba network. The idea is that depth images first go through a convolutional encoder to extract spatial features of the terrain, while proprioceptive signals (joint angles, velocities, body orientation, etc.) are processed in parallel. Then, following the FiLM (Feature-wise Linear Modulation) paradigm, the proprioceptive vector is used to compute affine scaling and shifting parameters that modulate the visual features. In practice, that means the vision feature maps are adaptively filtered by the current state: for example, if the robot is pitching forward heavily, the FiLM layers might emphasize ground features ahead and de-emphasize irrelevant background. By the authors’ description, this explicitly implements a visual–proprioceptive fusion at each time step. After fusing the modalities, the resulting feature vector is passed into a Mamba temporal backbone. Mamba refers to a recent class of structured state-space models that handle sequences very efficiently (with linear or even constant time per step). In other words, unlike a standard LSTM or Transformer which might have growing inference time with longer sequences, a Mamba-based SSM can process arbitrarily long histories cheaply. The use of Mamba is purposeful: for continuous, high-speed control you need bounded inference cost. As the paper notes, state-space models such as Mamba “provide bounded O(1) inference complexity” and outperform conventional recurrent or attention models in real-time settings. The net effect is that the student controller can maintain spatio-temporal memory of recent terrain – crucial, for example, if it saw an obstacle but is about to pass behind it or if vision suddenly goes dark, it can fall back on what it remembers. In summary, the student network uses proprioception to modulate vision features (via FiLM) and then feeds these into a fast sequence model (Mamba) that carries short-term memory. This architecture actively filters out unreliable visual cues (say, a brief glitch in the camera) and relies more on proprioception and memory instead. By distilling the teacher’s behavior into this network, REAL tightly couples vision, state, and time: the teacher’s cross-modal reasoning becomes embedded in the student’s FiLM+Mamba backbone. We highlight two practical details of spatio-temporal learning:. Privileged Teacher vs Onboard Student. During training, the teacher’s extra inputs (terrain scans, map, etc.) allow it to learn maneuvers that perfectly foot the unseen environment. The paper emphasizes: “the teacher has access to proprioceptive observations, terrain scan points, and additional privileged information, enabling it to learn precise proprioception–terrain associations for agile locomotion”. Then the student is taught by imitation. Concretely, the student is only given on-board depth images and proprioception (no map), yet it tries to match the teacher’s actions. This distillation is done in simulation to collect the data. As the authors state, “Stage 2 distills a deployable student policy that operates using only onboard depth sensing and proprioception”. Thus every time step in training pairs the teacher’s action (with full context) to the student’s observed action (with partial context), implicitly teaching the student how to infer hidden terrain from what it can actually see. Mamba Backbone and FiLM Fusion. The student’s core is the FiLM-modulated Mamba network. One way to think of this is that at every control step, the robot forms a latent state that captures what it believes about the recent terrain. If the vision is noisy, the Mamba remembers the last few frames and proprioceptive motion to smooth decision-making. If the robot has been running but then enters a tunnel (vision blackout), that latent memory carries forward. In the authors’ words, this yields “short-term spatio-temporal memory” that suppresses unreliable perception. (By contrast, prior parkour controllers often had no temporal depth – e.g. planning leaps from the current image alone – so they stumble when the current frame is wrong.). All told, this spatio-temporal learning component marries vision and proprioception with memory, providing a robust backbone for agility under imperfect sensing. Importantly, the student network’s design enables sub-time-step computation: using Mamba’s constant-time inference means the model can run well within real-time control limits. 2. Physics-Guided State Estimation. Fast breathing from the spatio-temporal policy is only one part of the story. The other critical thread is enforcing physical consistency, especially through the explosive dynamics of parkour. When a robot leaps and lands, small errors in estimated velocity or orientation can accumulate, and without correction, the policy might “forget” its place in space or generate jerky commands. REAL addresses this with a physics-guided filtering module – essentially a learned observer that fuses the robot’s sensed motion with a rigid-body model in an Extended Kalman Filter (EKF). Here’s how it works. The authors first train a lightweight neural network to predict the robot’s base velocity (and an associated uncertainty) from recent sensor inputs. In formula, you could view this as a function (v_{\text{pred}} = f(\text{sensors})) that outputs both a mean velocity and a confidence. This “neural velocity estimator” captures typical motion under locomotion (for example, given leg angles and IMU data, it guesses how fast the base is moving). Because it is learned from data, the network also outputs an adaptive uncertainty: if the situation is unusual or input noisy, it will report larger variance. This learned velocity and uncertainty then feed into a classical EKF that embodies the rigid-body dynamics of the quadruped. The EKF has a simple model: it knows that in between contact events, the robot’s base should follow ballistic motion, and when a foot is on the ground it should behave like an anchor (zero velocity relative to the ground). By fusing the noisy IMU and joint measurements with this model, the EKF smooths out noise. Crucially, the neural velocity provides an extra “pseudo-measurement” to the filter: it’s like telling the EKF, “I think the robot’s base was moving at (v_{\text{pred}}) with this confidence.” The EKF then updates the base pose estimate accordingly, always projecting back to a physically consistent motion. The benefit is two-fold: first, it handles abrupt impacts. For example, when a foot strikes the ground, raw IMU integration can glitch (even a brief overload). The filter instead enforces that at contact the velocity must rapidly adjust to satisfy the contact constraint, eliminating spurious jumps. Second, it smooths out drift during flight. In mid-air, the true physics might tell us the robot should decelerate upward then free-fall. The combined model-with-predictor expects those patterns, so even if the vision network’s foot placement predictions are momentarily off, the estimate stays plausible. As the paper notes, this “hybrid design enhances robustness to perceptual noise and prevents physically implausible state estimates during aggressive maneuvers”. In effect, the EKF anchors the robot’s state in reality. By onlining this state estimate into the policy loop, REAL ensures that its controller is always reasoning about a consistent body position and velocity. The policy does not just blindly react to raw proprioceptive streams; it plans with the filtered state. This physics-guided layer is especially important when vision is unreliable: for instance, if an obstacle is missed or mis-seen, the EKF will still keep the body pose tracking the last known motion instead of veering wildly. In short, it enforces the basic rigid-body laws underneath the learning. A key detail is that this velocity predictor-EKF combination is uncertainty-aware. When the vision is noisy or the robot is slipping, the predictor’s uncertainty grows, causing the EKF to rely more on the physical model. Conversely, when the robot is in control and the predictor is confident, the filter leans on the data. This dynamic fusion is highlighted by the authors: they fuse an “uncertainty-aware neural velocity estimator with rigid-body dynamics via an EKF” so that the filter automatically trusts the best available source. In experiments the effect is clear: the EKF-based state remains stable under impacts and yields smoother locomotion than a raw IMU chain would. 3. Consistency-Aware Training. Training this spatio-temporal, filtered controller in simulation raises its own challenges. In Stage 2, the student policy is distilled from the teacher via imitation, but the world is randomized (friction, mass, sensor delay, etc.) to prepare for the real robot. Under heavy domain randomization, simple cloning of the teacher’s actions can actually hurt: the student might follow the teacher too literally when the situations don’t match. Moreover, pure RL fine-tuning without teacher guidance can suffer from sparse reward and instability. REAL’s solution is a clever loss-gating strategy that dynamically blends behavioral cloning (BC) with reinforcement learning (RL) losses, depending on how well the student is tracking the teacher. In practice, they introduce a gating coefficient (\lambda\in [0,1]) that weights the RL loss versus the BC loss in each training step. The novelty is that (\lambda) is not fixed but adapted online based on the current disagreement between student and teacher. If the student’s action is very different from what the teacher would do (indicating the student’s understanding is poor), the gating turns up the imitation loss so the student re-aligns. If the student is close to the teacher (discrepancy is small), the gate shifts emphasis to RL, letting the student explore solutions tailored to its own sensor noise and dynamics. The paper describes it this way: they introduce a gating coefficient “to dynamically balance the RL loss and the behavioral cloning loss,” with the factor determined by the Euclidean distance between student and teacher actions. Put differently, this consistency-aware gating forces the student to match the teacher when it needs guidance, but slowly increases autonomy as training goes on. It prevents catastrophic collapse: if the student starts making erratic decisions under domain randomization, the imitation objective pulls it back. Conversely, it relaxes imitation when the student is already behaving well. The authors argue this adaptive scheme “stabilizes policy distillation and improves sim-to-real transfer” because it guards against the student “collapsing” under aggressive randomization (e.g. if the teacher’s privileged strategy is suddenly unrecoverable by the student). This is in contrast to a naive approach that might either always clone the teacher (leading to poor generalization) or always let RL roam (leading to divergence from the teacher’s skill). By measuring the consistency between the two and gating accordingly, the method smoothly transitions from imitation to reinforcement. In effect, the student’s learning is “aware” of when it’s being faithful to the teacher, and the loss shifts to correct only when mistakes occur. (It’s akin to the intuition behind some DAGGER or coach frameworks, but implemented as a soft adaptive weight.) Overall, these training mechanics ensure that the student eventually learns to handle the noisy world without forgetting what the teacher taught it about extreme maneuvers. 4. Experiments and Evaluation. The authors rigorously evaluated REAL both in simulation and on a physical robot. The simulation environments focus on “extreme parkour” layouts: sequences of hurdles, steps, and gaps that a quadruped must jump or traverse at high speed. Crucially, they impose perceptual challenges: for example, in some tests the depth camera is masked in the 1 meter immediately before each obstacle, simulating a sudden blind spot. They compare REAL against several recent baselines designed for agile locomotion:. Extreme Parkour (Cheng et al., 2023) – a state-of-the-art method for jumping obstacles, but without explicit memory. SoloParkour (Chane-Sane et al., CoRL 2024) – a prior teacher–student approach for visual locomotion, but using a different network and without the special filtering. RPL (Zhang et al., 2026) – a method for humanoid perceived locomotion. The metrics include Success Rate (did the robot complete all obstacles), Mean Distance progressed, Collision rate, and Edge violations. One striking result comes from the “vision-masked” tests (Table IV in the paper): with the camera blind for the last meter, REAL achieves a 55% success rate, far above SoloParkour’s 36% and the others’ near 0–11%. In other words, REAL traverses more than half of these blind maneuvers, whereas reasonable baselines rarely succeed. Additionally, REAL’s collisions and drift are far lower: its Mean Edge Violation is ~24 (much less than 103 for SoloParkour) and collision count is also lower. Even when the student’s Mamba backbone is ablated (“REAL w/o Mamba”), the success rate drops from 55% to 45%, confirming that the temporal memory is critical under blindness. On fully visual runs (no blind zone), REAL often doubles the success of prior methods on the hardest terrains, according to the authors. While the exact numbers are in a large table, the takeaway is consistent: the spatio-temporal policy and filtering let REAL handle terrain uncertainty that stumps other controllers. The physics filter is active too – in tests with impact and slip, REAL maintains low orientation error while others accumulate drift (the paper’s figures show clean trajectories for REAL vs jerky ones for baselines). Importantly, these results transfer to the real robot. The paper reports that on a real Unitree Go2 quadruped platform, REAL successfully crosses dynamic obstacle courses even with the blind zone, while a baseline controller fails immediately when vision is lost. One demo photo (captioned in the text) shows the REAL robot lunging over a set of boxes with no leading vision, holding form despite not seeing the obstacles until touchdown. Quantitatively, the inference speed is also demonstrated: REAL meets a tight 13.1 ms per-step inference time constraint, asleep within high-frequency control loops. Finally, note how all three core ideas appear necessary. A version of REAL without the Mamba backbone or without the guided filter performs noticeably worse in the ablations. And training without the loss gate often led to policies that overfit the simulator or oscillated during learning. The combined package passes every check. The authors even provide a breakdown of worst-case conditions (e.g. severe Gaussian noise or random delays) where REAL degrades gracefully, whereas simpler policies completely fail as soon as their vision is perturbed. Conclusion. In summary, REAL delivers an elegant recipe for making parkour-capable robots robust to the messiness of real sensor data. The insight is that agility under uncertainty requires memory and consistency, not just raw speed. By distilling a privileged teacher into a FiLM-Mamba student, embedding short-term terrain memory, and then bolstering the perception pipeline with a physics-aware state filter, the method constructs a control policy that can actually “ride out” brief blips in the vision. At the very least, this is a powerful demonstration that tasks like legged parkour – with all their dynamic impacts – can be tackled in the wild, not just in perfect sims. For practitioners, the takeaway is practical: one can incorporate state-space memory models (like Mamba) and learned filtering into existing RL locomotion frameworks to handle noise. And the consistency-based distillation shows a way to mix imitation learning with RL in a stable fashion. Going forward, such spatio-temporal and physics-informed policies could extend to other challenging scenarios – imagine bulldozing through smoke, fog, or dust as long as the policy has seen similar cues and learned to trust its history. The real robot demonstrations here suggest that within the next few years we’ll see quadrupeds vaulting unstructured rubble and rubble-ruined halls with the same confidence as they already do in pristine labs – and maybe even doing it blindfolded for a moment, thanks to techniques like those in REAL.