REAL: Robust Extreme Agility via Spatio-Temporal Policy Learning and Physics-Guided Filtering

TL;DR

This paper introduces REAL, a robust learning framework for quadruped robots that combines multi-modal perception, physics-guided filtering, and memory to achieve extreme agility under sensory noise and perceptual degradation.

Contribution

REAL integrates vision, proprioception, and physics-based filtering into a unified system, improving robustness and reliability in dynamic, obstacle-rich environments.

Findings

Successfully traverses extreme obstacles with visual noise

Operates in real-time with 13.1 ms inference time

Maintains high agility despite sensory corruption

Abstract

Extreme legged parkour demands rapid terrain assessment and precise foot placement under highly dynamic conditions. While recent learning-based systems achieve impressive agility, they remain fundamentally fragile to perceptual degradation, where even brief visual noise or latency can cause catastrophic failure. To overcome this, we propose Robust Extreme Agility Learning (REAL), an end-to-end framework for reliable parkour under sensory corruption. Instead of relying on perfectly clean perception, REAL tightly couples vision, proprioceptive history, and temporal memory. We distill a cross-modal teacher policy into a deployable student equipped with a FiLM-modulated Mamba backbone to actively filter visual noise and build short-term terrain memory actively. Furthermore, a physics-guided Bayesian state estimator enforces rigid-body consistency during high-impact maneuvers. Validated on a Unitree Go2 quadruped, REAL successfully traverses extreme obstacles even with a 1-meter visual blind zone, while strictly satisfying real-time control constraints with a bounded 13.1 ms inference time.