Learning Natural and Robust Hexapod Locomotion over Complex Terrains via Motion Priors based on Deep Reinforcement Learning

TL;DR

This paper presents a novel deep reinforcement learning approach using motion priors to enable a hexapod robot to learn natural and robust walking gaits over complex terrains, successfully transferring from simulation to real-world operation.

Contribution

Introducing a motion prior-based deep reinforcement learning method for real hexapod locomotion on complex terrains, achieving natural gait patterns and robustness without visual cues.

Findings

Successful transfer of learned policies to real robot

Natural gait patterns achieved in complex terrains

Robust locomotion without reliance on visual information

Abstract

Multi-legged robots offer enhanced stability to navigate complex terrains with their multiple legs interacting with the environment. However, how to effectively coordinate the multiple legs in a larger action exploration space to generate natural and robust movements is a key issue. In this paper, we introduce a motion prior-based approach, successfully applying deep reinforcement learning algorithms to a real hexapod robot. We generate a dataset of optimized motion priors, and train an adversarial discriminator based on the priors to guide the hexapod robot to learn natural gaits. The learned policy is then successfully transferred to a real hexapod robot, and demonstrate natural gait patterns and remarkable robustness without visual information in complex terrains. This is the first time that a reinforcement learning controller has been used to achieve complex terrain walking on a real hexapod robot.