Learning Agile Locomotion on Risky Terrains

TL;DR

This paper demonstrates that end-to-end reinforcement learning enables quadruped robots to traverse risky terrains like stepping stones and balance beams, achieving high speeds and robustness through a transfer learning approach.

Contribution

It introduces a transfer learning framework for reinforcement learning that trains a generalist policy and fine-tunes specialist policies for challenging terrains, improving adaptability and performance.

Findings

Achieved >= 2.5 m/s speed on sparse stepping stones

Validated transfer learning approach in simulation and real-world

Enhanced robustness in navigating risky terrains

Abstract

Quadruped robots have shown remarkable mobility on various terrains through reinforcement learning. Yet, in the presence of sparse footholds and risky terrains such as stepping stones and balance beams, which require precise foot placement to avoid falls, model-based approaches are often used. In this paper, we show that end-to-end reinforcement learning can also enable the robot to traverse risky terrains with dynamic motions. To this end, our approach involves training a generalist policy for agile locomotion on disorderly and sparse stepping stones before transferring its reusable knowledge to various more challenging terrains by finetuning specialist policies from it. Given that the robot needs to rapidly adapt its velocity on these terrains, we formulate the task as a navigation task instead of the commonly used velocity tracking which constrains the robot's behavior and propose an exploration strategy to overcome sparse rewards and achieve high robustness. We validate our proposed method through simulation and real-world experiments on an ANYmal-D robot achieving peak forward velocity of >= 2.5 m/s on sparse stepping stones and narrow balance beams. Video: youtu.be/Z5X0J8OH6z4