DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning

TL;DR

This paper presents a novel deep reinforcement learning method enabling omnidirectional bipedal locomotion in humanoid robots using a single neural network policy, with curriculum learning and sim-to-real transfer strategies.

Contribution

It introduces a curriculum learning approach for omnidirectional gait learning without reference motions, adaptable to various robot kinematics.

Findings

Successful transfer of learned policy to real humanoid robot

Single neural network controls omnidirectional gait

Effective curriculum learning improves training efficiency

Abstract

Bipedal walking is one of the most difficult but exciting challenges in robotics. The difficulties arise from the complexity of high-dimensional dynamics, sensing and actuation limitations combined with real-time and computational constraints. Deep Reinforcement Learning (DRL) holds the promise to address these issues by fully exploiting the robot dynamics with minimal craftsmanship. In this paper, we propose a novel DRL approach that enables an agent to learn omnidirectional locomotion for humanoid (bipedal) robots. Notably, the locomotion behaviors are accomplished by a single control policy (a single neural network). We achieve this by introducing a new curriculum learning method that gradually increases the task difficulty by scheduling target velocities. In addition, our method does not require reference motions which facilities its application to robots with different kinematics, and reduces the overall complexity. Finally, different strategies for sim-to-real transfer are presented which allow us to transfer the learned policy to a real humanoid robot.