Learning Dexterous In-Hand Manipulation

TL;DR

This paper presents a reinforcement learning approach to teach a robotic hand dexterous in-hand manipulation skills in simulation, which successfully transfer to real-world robots without human demonstrations.

Contribution

The authors develop a simulation-based RL method that enables a robotic hand to learn complex manipulation behaviors that transfer to physical hardware without demonstrations.

Findings

Policies transfer successfully from simulation to real robot

Emergence of human-like manipulation behaviors such as finger gaiting

Use of distributed RL system similar to training OpenAI Five

Abstract

We use reinforcement learning (RL) to learn dexterous in-hand manipulation policies which can perform vision-based object reorientation on a physical Shadow Dexterous Hand. The training is performed in a simulated environment in which we randomize many of the physical properties of the system like friction coefficients and an object's appearance. Our policies transfer to the physical robot despite being trained entirely in simulation. Our method does not rely on any human demonstrations, but many behaviors found in human manipulation emerge naturally, including finger gaiting, multi-finger coordination, and the controlled use of gravity. Our results were obtained using the same distributed RL system that was used to train OpenAI Five. We also include a video of our results: https://youtu.be/jwSbzNHGflM