Robotic Arm Control and Task Training through Deep Reinforcement Learning

TL;DR

This paper compares advanced deep reinforcement learning algorithms for robotic arm control, demonstrating that Trust Region Policy Optimization and DeepQ-Network with Normalized Advantage Functions outperform other methods in simulation and real-world tasks.

Contribution

It provides a comprehensive comparison of RL algorithms for robotic manipulation, including procedures for hyper-parameter estimation and policy transfer from simulation to real-world.

Findings

TRPO and DQN-NAF outperform DDPG and VPG in manipulation tasks

Policies trained in simulation transfer effectively to real robots

Simulation-based hyper-parameter tuning improves real-world performance

Abstract

This paper proposes a detailed and extensive comparison of the Trust Region Policy Optimization and DeepQ-Network with Normalized Advantage Functions with respect to other state of the art algorithms, namely Deep Deterministic Policy Gradient and Vanilla Policy Gradient. Comparisons demonstrate that the former have better performances then the latter when asking robotic arms to accomplish manipulation tasks such as reaching a random target pose and pick &placing an object. Both simulated and real-world experiments are provided. Simulation lets us show the procedures that we adopted to precisely estimate the algorithms hyper-parameters and to correctly design good policies. Real-world experiments let show that our polices, if correctly trained on simulation, can be transferred and executed in a real environment with almost no changes.