Learning Vision-Guided Dynamic Locomotion Over Challenging Terrains

TL;DR

This paper introduces a deep reinforcement learning method for legged robots that uses Lidar perception and a novel Dynamic Reward Strategy to achieve robust locomotion over difficult terrains with high success rates.

Contribution

It presents a new learning strategy, DRS, enabling versatile gait learning without history data, and demonstrates high success in challenging terrains.

Findings

Over 90% success rate in tested terrains

Effective Lidar-based perceptual policy learned via PPO

Novel Dynamic Reward Strategy improves gait versatility

Abstract

Legged robots are becoming increasingly powerful and popular in recent years for their potential to bring the mobility of autonomous agents to the next level. This work presents a deep reinforcement learning approach that learns a robust Lidar-based perceptual locomotion policy in a partially observable environment using Proximal Policy Optimisation. Visual perception is critical to actively overcome challenging terrains, and to do so, we propose a novel learning strategy: Dynamic Reward Strategy (DRS), which serves as effective heuristics to learn a versatile gait using a neural network architecture without the need to access the history data. Moreover, in a modified version of the OpenAI gym environment, the proposed work is evaluated with scores over 90% success rate in all tested challenging terrains.