RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains

TL;DR

This paper introduces RPL, a two-stage training framework that enables humanoid robots to perform robust multi-directional locomotion on complex terrains with payloads, using terrain-specific policies and a transformer-based distillation approach.

Contribution

The paper presents a novel two-stage training framework with techniques for robust multi-directional locomotion on challenging terrains, including efficient multi-depth simulation and depth feature scaling methods.

Findings

Achieves robust locomotion on complex terrains with payloads.

Speeds up depth rendering by 5 times in simulation.

Demonstrates successful real-world experiments on various challenging terrains.

Abstract

Humanoid perceptive locomotion has made significant progress and shows great promise, yet achieving robust multi-directional locomotion on complex terrains remains underexplored. To tackle this challenge, we propose RPL, a two-stage training framework that enables multi-directional locomotion on challenging terrains, and remains robust with payloads. RPL first trains terrain-specific expert policies with privileged height map observations to master decoupled locomotion and manipulation skills across different terrains, and then distills them into a transformer policy that leverages multiple depth cameras to cover a wide range of views. During distillation, we introduce two techniques to robustify multi-directional locomotion, depth feature scaling based on velocity commands and random side masking, which are critical for asymmetric depth observations and unseen widths of terrains. For scalable depth distillation, we develop an efficient multi-depth system that ray-casts against both dynamic robot meshes and static terrain meshes in massively parallel environments, achieving a 5-times speedup over the depth rendering pipelines in existing simulators while modeling realistic sensor latency, noise, and dropout. Extensive real-world experiments demonstrate robust multi-directional locomotion with payloads (2kg) across challenging terrains, including 20{\deg} slopes, staircases with different step lengths (22 cm, 25 cm, 30 cm), and 25 cm by 25 cm stepping stones separated by 60 cm gaps.