RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

TL;DR

This paper introduces RLOC, a combined model-based and data-driven approach for quadrupedal robots to achieve stable, terrain-aware locomotion over uneven surfaces using reinforcement learning and optimal control.

Contribution

It presents a unified framework integrating RL and model-based control for terrain-adaptive quadrupedal locomotion, including transferability to different robot models.

Findings

Robust locomotion over diverse complex terrains.

Successful transfer of policies between different robot models.

Enhanced stability through ancillary RL policies.

Abstract

We present a unified model-based and data-driven approach for quadrupedal planning and control to achieve dynamic locomotion over uneven terrain. We utilize on-board proprioceptive and exteroceptive feedback to map sensory information and desired base velocity commands into footstep plans using a reinforcement learning (RL) policy. This RL policy is trained in simulation over a wide range of procedurally generated terrains. When ran online, the system tracks the generated footstep plans using a model-based motion controller. We evaluate the robustness of our method over a wide variety of complex terrains. It exhibits behaviors which prioritize stability over aggressive locomotion. Additionally, we introduce two ancillary RL policies for corrective whole-body motion tracking and recovery control. These policies account for changes in physical parameters and external perturbations. We train and evaluate our framework on a complex quadrupedal system, ANYmal version B, and demonstrate transferability to a larger and heavier robot, ANYmal C, without requiring retraining.