Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Control

TL;DR

This paper presents a novel vision-based deep reinforcement learning system enabling a robotic manipulator to learn target reaching solely from raw visual input, demonstrating initial success in simulation and synthetic image transfer to real hardware.

Contribution

It introduces a method for training a robot controller from raw images without prior configuration knowledge, highlighting the potential of synthetic images for real-world transfer.

Findings

Deep Q Network successfully learned target reaching in simulation.

Naive transfer to real hardware failed with real images.

Synthetic images enabled successful transfer to real robot.

Abstract

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.