Quadruped Robot Simulation Using Deep Reinforcement Learning -- A step towards locomotion policy

TL;DR

This paper introduces a novel deep reinforcement learning approach to train quadruped robots in simulation, achieving efficient policy learning with limited resources and providing a foundation for further research in legged robot control.

Contribution

The paper presents a new reinforcement learning method using proximal policy optimization for quadruped robot simulation, demonstrating effective training with limited computational resources.

Findings

Achieved successful policy training in simulation using a desktop machine.

Utilized open-source and proprietary simulation tools for realistic robot modeling.

Provided a framework for early-stage researchers to deploy reinforcement learning algorithms.

Abstract

We present a novel reinforcement learning method to train the quadruped robot in a simulated environment. The idea of controlling quadruped robots in a dynamic environment is quite challenging and my method presents the optimum policy and training scheme with limited resources and shows considerable performance. The report uses the raisimGymTorch open-source library and proprietary software RaiSim for the simulation of ANYmal robot. My approach is centered on formulating Markov decision processes using the evaluation of the robot walking scheme while training. Resulting MDPs are solved using a proximal policy optimization algorithm used in actor-critic mode and collected thousands of state transitions with a single desktop machine. This work also presents a controller scheme trained over thousands of time steps shown in a simulated environment. This work also sets the base for early-stage researchers to deploy their favorite algorithms and configurations. Keywords: Legged robots, deep reinforcement learning, quadruped robot simulation, optimal control