Beyond Egocentric Limits: Multi-View Depth-Based Learning for Robust Quadrupedal Locomotion

TL;DR

This paper introduces a multi-view depth-based learning framework for quadrupedal robots that combines egocentric and exocentric perceptions, significantly enhancing robustness and agility in dynamic locomotion tasks.

Contribution

It proposes a novel multi-view depth perception approach with a teacher-student training scheme and extensive domain randomization to improve robustness in quadrupedal locomotion.

Findings

Multi-view policies outperform single-view baselines in dynamic tasks.

The approach maintains stability despite partial or full exocentric camera failure.

Moderate viewpoint misalignment is tolerated when included during training.

Abstract

Recent progress in legged locomotion has allowed highly dynamic and parkour-like behaviors for robots, similar to their biological counterparts. Yet, these methods mostly rely on egocentric (first-person) perception, limiting their performance, especially when the viewpoint of the robot is occluded. A promising solution would be to enhance the robot's environmental awareness by using complementary viewpoints, such as multiple actors exchanging perceptual information. Inspired by this idea, this work proposes a multi-view depth-based locomotion framework that combines egocentric and exocentric observations to provide richer environmental context during agile locomotion. Using a teacher-student distillation approach, the student policy learns to fuse proprioception with dual depth streams while remaining robust to real-world sensing imperfections. To further improve robustness, we introduce extensive domain randomization, including stochastic remote-camera dropouts and 3D positional perturbations that emulate aerial-ground cooperative sensing. Simulation results show that multi-viewpoints policies outperform single-viewpoint baseline in gap crossing, step descent, and other dynamic maneuvers, while maintaining stability when the exocentric camera is partially or completely unavailable. Additional experiments show that moderate viewpoint misalignment is well tolerated when incorporated during training. This study demonstrates that heterogeneous visual feedback improves robustness and agility in quadrupedal locomotion. Furthermore, to support reproducibility, the implementation accompanying this work is publicly available at https://anonymous.4open.science/r/multiview-parkour-6FB8