Learning Purely Tactile In-Hand Manipulation with a Torque-Controlled Hand

TL;DR

This paper demonstrates that a humanoid robotic hand can learn and perform purely tactile in-hand cube manipulation using deep reinforcement learning, without visual input, achieving robust real-world performance.

Contribution

It introduces a method for learning tactile in-hand manipulation solely from tactile sensing, with successful transfer from simulation to a real humanoid hand.

Findings

Robust transfer of learned policy to real robot.

Achieved over 46 full rotations of the cube in a single run.

Policy is resilient to disturbances like different cube sizes and hand orientations.

Abstract

We show that a purely tactile dextrous in-hand manipulation task with continuous regrasping, requiring permanent force closure, can be learned from scratch and executed robustly on a torque-controlled humanoid robotic hand. The task is rotating a cube without dropping it, but in contrast to OpenAI's seminal cube manipulation task, the palm faces downwards and no cameras but only the hand's position and torque sensing are used. Although the task seems simple, it combines for the first time all the challenges in execution as well as learning that are important for using in-hand manipulation in real-world applications. We efficiently train in a precisely modeled and identified rigid body simulation with off-policy deep reinforcement learning, significantly sped up by a domain adapted curriculum, leading to a moderate 600 CPU hours of training time. The resulting policy is robustly transferred to the real humanoid DLR Hand-II, e.g., reaching more than 46 full 2${\pi}$ rotations of the cube in a single run and allowing for disturbances like different cube sizes, hand orientation, or pulling a finger.