Learning Quadrupedal Locomotion over Challenging Terrain

TL;DR

This paper introduces a neural network-based reinforcement learning controller for quadrupedal robots, enabling robust, zero-shot generalization to challenging natural terrains like mud, snow, and water, surpassing traditional methods.

Contribution

The work presents a novel proprioception-based reinforcement learning approach that achieves remarkable robustness and generalization in natural environments for legged robots.

Findings

Successful transfer from simulation to real-world environments

Robust performance on deformable and dynamic terrains

Zero-shot generalization to unseen challenging terrains

Abstract

Some of the most challenging environments on our planet are accessible to quadrupedal animals but remain out of reach for autonomous machines. Legged locomotion can dramatically expand the operational domains of robotics. However, conventional controllers for legged locomotion are based on elaborate state machines that explicitly trigger the execution of motion primitives and reflexes. These designs have escalated in complexity while falling short of the generality and robustness of animal locomotion. Here we present a radically robust controller for legged locomotion in challenging natural environments. We present a novel solution to incorporating proprioceptive feedback in locomotion control and demonstrate remarkable zero-shot generalization from simulation to natural environments. The controller is trained by reinforcement learning in simulation. It is based on a neural network that acts on a stream of proprioceptive signals. The trained controller has taken two generations of quadrupedal ANYmal robots to a variety of natural environments that are beyond the reach of prior published work in legged locomotion. The controller retains its robustness under conditions that have never been encountered during training: deformable terrain such as mud and snow, dynamic footholds such as rubble, and overground impediments such as thick vegetation and gushing water. The presented work opens new frontiers for robotics and indicates that radical robustness in natural environments can be achieved by training in much simpler domains.