Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

TL;DR

This paper presents a reinforcement learning approach for humanoid robots to follow planned footsteps, enabling robust omnidirectional walking, turning, and stair climbing in simulation without pre-trained models.

Contribution

It introduces a policy trained on procedural footstep plans that achieves versatile walking behaviors, applicable to multiple robot platforms without reference motions.

Findings

Successful omnidirectional walking and turning in simulation

Effective stair climbing capabilities

Applicability to multiple humanoid robot platforms

Abstract

Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called footstep plans). We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs curriculum learning on the complexity of terrains, and circumvents the need for reference motions or pre-trained weights. We demonstrate the application of our proposed method to learn RL policies for 2 new robot platforms - HRP5P and JVRC-1 - in the MuJoCo simulation environment. The code for training and evaluation is available online.