Reinforcement Learning Experiments and Benchmark for Solving Robotic Reaching Tasks

TL;DR

This paper establishes a systematic benchmark for evaluating model-free reinforcement learning algorithms on robotic reaching tasks, demonstrating that Hindsight Experience Replay significantly improves off-policy learning performance in simulation-to-real transfer scenarios.

Contribution

It introduces a reproducible experimental framework for comparing RL algorithms on robotic reaching tasks and highlights the effectiveness of Hindsight Experience Replay in enhancing off-policy learning.

Findings

Hindsight Experience Replay increases average return 7-9 times.

Systematic comparison of RL algorithms in simulation and real robots.

Reproducible procedures for robotic reaching benchmarks.

Abstract

Reinforcement learning has shown great promise in robotics thanks to its ability to develop efficient robotic control procedures through self-training. In particular, reinforcement learning has been successfully applied to solving the reaching task with robotic arms. In this paper, we define a robust, reproducible and systematic experimental procedure to compare the performance of various model-free algorithms at solving this task. The policies are trained in simulation and are then transferred to a physical robotic manipulator. It is shown that augmenting the reward signal with the Hindsight Experience Replay exploration technique increases the average return of off-policy agents between 7 and 9 folds when the target position is initialised randomly at the beginning of each episode.