Bipedal Walking Robot using Deep Deterministic Policy Gradient

TL;DR

This paper presents a reinforcement learning approach using Deep Deterministic Policy Gradient to enable a simulated bipedal robot to learn stable walking and running behaviors autonomously, with gait patterns similar to humans.

Contribution

It introduces a novel application of DDPG for training a bipedal robot in simulation, achieving human-like gait patterns without prior knowledge of dynamics.

Findings

Robot achieved an average speed of 0.83 m/s.

Gait patterns closely resembled human walking.

Successful autonomous learning through trial and error.

Abstract

Machine learning algorithms have found several applications in the field of robotics and control systems. The control systems community has started to show interest towards several machine learning algorithms from the sub-domains such as supervised learning, imitation learning and reinforcement learning to achieve autonomous control and intelligent decision making. Amongst many complex control problems, stable bipedal walking has been the most challenging problem. In this paper, we present an architecture to design and simulate a planar bipedal walking robot(BWR) using a realistic robotics simulator, Gazebo. The robot demonstrates successful walking behaviour by learning through several of its trial and errors, without any prior knowledge of itself or the world dynamics. The autonomous walking of the BWR is achieved using reinforcement learning algorithm called Deep Deterministic Policy Gradient(DDPG). DDPG is one of the algorithms for learning controls in continuous action spaces. After training the model in simulation, it was observed that, with a proper shaped reward function, the robot achieved faster walking or even rendered a running gait with an average speed of 0.83 m/s. The gait pattern of the bipedal walker was compared with the actual human walking pattern. The results show that the bipedal walking pattern had similar characteristics to that of a human walking pattern. The video presenting our experiment is available at https://goo.gl/NHXKqR.