Learning Agile Locomotion Skills with a Mentor

TL;DR

This paper introduces a multi-stage reinforcement learning approach where a mentor guides a quadruped robot to develop agile locomotion skills, significantly improving performance over traditional methods.

Contribution

It proposes a novel mentor-guided multi-stage learning framework for agile robot locomotion, reducing the need for reward shaping and curriculum design.

Findings

Outperforms single-stage RL baseline in agility tasks

Enables quadruped to run and jump across gaps and obstacles

Provides detailed analysis of learned behaviors' feasibility

Abstract

Developing agile behaviors for legged robots remains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot's center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform the task without the mentor. We evaluate our proposed learning system with a simulated quadruped robot on a course consisting of randomly generated gaps and hurdles. Our method significantly outperforms a single-stage RL baseline without a mentor, and the quadruped robot can agilely run and jump across gaps and obstacles. Finally, we present a detailed analysis of the learned behaviors' feasibility and efficiency.