START: Traversing Sparse Footholds with Terrain Reconstruction

TL;DR

START is a novel single-stage learning framework that enables quadruped robots to traverse highly sparse terrains using onboard vision and proprioception for accurate terrain reconstruction, improving adaptability and stability.

Contribution

It introduces a new approach that explicitly reconstructs terrain heightmaps from onboard sensors, enhancing environmental understanding and locomotion on sparse footholds.

Findings

Achieves zero-shot transfer in real-world scenarios

Demonstrates superior adaptability and robustness

Enables precise foothold placement on sparse terrains

Abstract

Traversing terrains with sparse footholds like legged animals presents a promising yet challenging task for quadruped robots, as it requires precise environmental perception and agile control to secure safe foot placement while maintaining dynamic stability. Model-based hierarchical controllers excel in laboratory settings, but suffer from limited generalization and overly conservative behaviors. End-to-end learning-based approaches unlock greater flexibility and adaptability, but existing state-of-the-art methods either rely on heightmaps that introduce noise and complex, costly pipelines, or implicitly infer terrain features from egocentric depth images, often missing accurate critical geometric cues and leading to inefficient learning and rigid gaits. To overcome these limitations, we propose START, a single-stage learning framework that enables agile, stable locomotion on highly sparse and randomized footholds. START leverages only low-cost onboard vision and proprioception to accurately reconstruct local terrain heightmap, providing an explicit intermediate representation to convey essential features relevant to sparse foothold regions. This supports comprehensive environmental understanding and precise terrain assessment, reducing exploration cost and accelerating skill acquisition. Experimental results demonstrate that START achieves zero-shot transfer across diverse real-world scenarios, showcasing superior adaptability, precise foothold placement, and robust locomotion.