BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds

TL;DR

BeamDojo is a reinforcement learning framework that enables humanoid robots to traverse complex terrains with sparse footholds by combining novel reward functions, a two-stage training process, and real-world LiDAR integration.

Contribution

The paper introduces BeamDojo, a new RL framework with a polygonal foot reward, double critic, and two-stage training for agile humanoid locomotion on sparse footholds.

Findings

Efficient learning in simulation for sparse foothold terrains.

Successful real-world deployment with precise foot placement.

High success rate under external disturbances.

Abstract

Traversing risky terrains with sparse footholds poses a significant challenge for humanoid robots, requiring precise foot placements and stable locomotion. Existing learning-based approaches often struggle on such complex terrains due to sparse foothold rewards and inefficient learning processes. To address these challenges, we introduce BeamDojo, a reinforcement learning (RL) framework designed for enabling agile humanoid locomotion on sparse footholds. BeamDojo begins by introducing a sampling-based foothold reward tailored for polygonal feet, along with a double critic to balancing the learning process between dense locomotion rewards and sparse foothold rewards. To encourage sufficient trial-and-error exploration, BeamDojo incorporates a two-stage RL approach: the first stage relaxes the terrain dynamics by training the humanoid on flat terrain while providing it with task-terrain perceptive observations, and the second stage fine-tunes the policy on the actual task terrain. Moreover, we implement a onboard LiDAR-based elevation map to enable real-world deployment. Extensive simulation and real-world experiments demonstrate that BeamDojo achieves efficient learning in simulation and enables agile locomotion with precise foot placement on sparse footholds in the real world, maintaining a high success rate even under significant external disturbances.