Terrain Consistent Reference-Guided RL for Humanoid Navigation Autonomy

TL;DR

This paper introduces a terrain-aware reinforcement learning method for humanoid robot navigation, enabling long-distance outdoor autonomy with terrain-consistent reference trajectories and integration with standard navigation systems.

Contribution

The authors develop a novel terrain-conditioned reference-guided RL approach that improves locomotion and navigation performance in complex outdoor environments.

Findings

Environmentally-conditioned references outperform environment-agnostic ones in simulation.

The method achieves over 70 meters of autonomous outdoor navigation on hardware.

The policy integrates seamlessly with standard navigation infrastructure.

Abstract

We present a method for training reference-guided, perceptive reinforcement learning locomotion policies for humanoid robots in which reference trajectories are modulated in training to be consistent with terrain geometry. Aiming to deploy our method with standard navigation autonomy infrastructure, we synthesize SE(2)-controllable reference trajectories inside the RL training loop, projecting desired footsteps onto valid footholds and adjusting swing-foot and center-of-mass trajectories to match the terrain. The resulting policy exposes a clean SE(2) velocity interface compatible with standard navigation planners. In simulation, environmentally-conditioned references significantly improve reference tracking performance compared to environment agnostic references. On hardware, we integrate the policy with an MPC + control barrier function planner and demonstrate long-horizon (>70m) closed-loop autonomous navigation on the Unitree G1 through outdoor environments containing rough terrain and consecutive flights of stairs, with all sensing and computation onboard.