Learning fast and agile quadrupedal locomotion over complex terrain

TL;DR

This paper introduces a reinforcement learning-based controller enabling a blind quadruped robot to achieve fast, natural, and stable locomotion over complex terrains using only proprioceptive data, with strong transferability to real robots.

Contribution

The paper presents a loose neighborhood control architecture and mirror-world neural network to improve learning efficiency, transferability, and performance balance in quadrupedal locomotion control.

Findings

Robot reaches speeds up to 10 times its body length.

Controller demonstrates excellent anti-disturbance and generalization abilities.

Natural gait patterns outperform artificially designed controllers.

Abstract

In this paper, we propose a robust controller that achieves natural and stably fast locomotion on a real blind quadruped robot. With only proprioceptive information, the quadruped robot can move at a maximum speed of 10 times its body length, and has the ability to pass through various complex terrains. The controller is trained in the simulation environment by model-free reinforcement learning. In this paper, the proposed loose neighborhood control architecture not only guarantees the learning rate, but also obtains an action network that is easy to transfer to a real quadruped robot. Our research finds that there is a problem of data symmetry loss during training, which leads to unbalanced performance of the learned controller on the left-right symmetric quadruped robot structure, and proposes a mirror-world neural network to solve the performance problem. The learned controller composed of the mirror-world network can make the robot achieve excellent anti-disturbance ability. No specific human knowledge such as a foot trajectory generator are used in the training architecture. The learned controller can coordinate the robot's gait frequency and locomotion speed, and the locomotion pattern is more natural and reasonable than the artificially designed controller. Our controller has excellent anti-disturbance performance, and has good generalization ability to reach locomotion speeds it has never learned and traverse terrains it has never seen before.