Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

TL;DR

This paper introduces a two-stage learning framework inspired by animals, enabling legged robots to efficiently learn natural and robust gaits through experience-based rewards, improving performance on complex terrains.

Contribution

A novel two-step reward-based learning method that mimics animal learning, significantly enhancing natural gait acquisition and robustness in legged robots across terrains.

Findings

Robots learned natural, robust gaits on flat terrain.

Successful transfer of policies to physical robots.

Robots demonstrated robustness in complex terrains.

Abstract

Multi-legged robots offer enhanced stability in complex terrains, yet autonomously learning natural and robust motions in such environments remains challenging. Drawing inspiration from animals' progressive learning patterns, from simple to complex tasks, we introduce a universal two-stage learning framework with two-step reward setting based on self-acquired experience, which efficiently enables legged robots to incrementally learn natural and robust movements. In the first stage, robots learn through gait-related rewards to track velocity on flat terrain, acquiring natural, robust movements and generating effective motion experience data. In the second stage, mirroring animal learning from existing experiences, robots learn to navigate challenging terrains with natural and robust movements using adversarial imitation learning. To demonstrate our method's efficacy, we trained both quadruped robots and a hexapod robot, and the policy were successfully transferred to a physical quadruped robot GO1, which exhibited natural gait patterns and remarkable robustness in various terrains.