Gait-Conditioned Reinforcement Learning with Multi-Phase Curriculum for Humanoid Locomotion

TL;DR

This paper introduces a unified reinforcement learning framework for humanoid robots that enables natural, stable, and versatile locomotion across multiple gaits and transitions, both in simulation and real-world settings.

Contribution

It proposes a gait-conditioned RL approach with a dynamic reward routing mechanism and a structured curriculum, supporting multi-gait learning without motion capture data.

Findings

Successful simulation of multiple gaits and transitions.

Real-world validation on Unitree G1 demonstrating stable locomotion.

Scalable, reference-free approach for humanoid control.

Abstract

We present a unified gait-conditioned reinforcement learning framework that enables humanoid robots to perform standing, walking, running, and smooth transitions within a single recurrent policy. A compact reward routing mechanism dynamically activates gait-specific objectives based on a one-hot gait ID, mitigating reward interference and supporting stable multi-gait learning. Human-inspired reward terms promote biomechanically natural motions, such as straight-knee stance and coordinated arm-leg swing, without requiring motion capture data. A structured curriculum progressively introduces gait complexity and expands command space over multiple phases. In simulation, the policy successfully achieves robust standing, walking, running, and gait transitions. On the real Unitree G1 humanoid, we validate standing, walking, and walk-to-stand transitions, demonstrating stable and coordinated locomotion. This work provides a scalable, reference-free solution toward versatile and naturalistic humanoid control across diverse modes and environments.