Scaling Rough Terrain Locomotion with Automatic Curriculum Reinforcement Learning

TL;DR

This paper introduces LP-ACRL, an automatic curriculum learning framework that adapts task difficulty based on learning progress, enabling quadruped robots to perform high-speed locomotion across diverse complex terrains.

Contribution

The paper presents a novel learning progress-based approach for automatic curriculum generation in reinforcement learning, addressing challenges in complex, unstructured task spaces.

Findings

Enables quadruped to walk at 2.5 m/s on various terrains

Demonstrates strong scalability and real-world applicability

Outperforms previous methods on complex terrains

Abstract

Curriculum learning has demonstrated substantial effectiveness in robot learning. However, it still faces limitations when scaling to complex, wide-ranging task spaces. Such task spaces often lack a well-defined difficulty structure, making the difficulty ordering required by previous methods challenging to define. We propose a Learning Progress-based Automatic Curriculum Reinforcement Learning (LP-ACRL) framework, which estimates the agent's learning progress online and adaptively adjusts the task-sampling distribution, thereby enabling automatic curriculum generation without prior knowledge of the difficulty distribution over the task space. Policies trained with LP-ACRL enable the ANYmal D quadruped to achieve and maintain stable, high-speed locomotion at 2.5 m/s linear velocity and 3.0 rad/s angular velocity across diverse terrains, including stairs, slopes, gravel, and low-friction flat surfaces--whereas previous methods have generally been limited to high speeds on flat terrain or low speeds on complex terrain. Experimental results demonstrate that LP-ACRL exhibits strong scalability and real-world applicability, providing a robust baseline for future research on curriculum generation in complex, wide-ranging robotic learning task spaces.