Fine Manipulation Using a Tactile Skin: Learning in Simulation and Sim-to-Real Transfer

TL;DR

This paper introduces a novel tactile skin model for robotic fingers, enabling precise manipulation tasks in simulation and successful transfer to real robots, advancing tactile-based robotic control.

Contribution

A new tactile skin model that accurately simulates softness and contact spread, with a calibration method, facilitating sim-to-real transfer for fine manipulation tasks.

Findings

Tactile feedback improves manipulation precision.

Policies transfer successfully from simulation to real robot.

Achieved sub-taxel resolution of less than 1 mm.

Abstract

We want to enable fine manipulation with a multi-fingered robotic hand by using modern deep reinforcement learning methods. Key for fine manipulation is a spatially resolved tactile sensor. Here, we present a novel model of a tactile skin that can be used together with rigid-body (hence fast) physics simulators. The model considers the softness of the real fingertips such that a contact can spread across multiple taxels of the sensor depending on the contact geometry. We calibrate the model parameters to allow for an accurate simulation of the real-world sensor. For this, we present a self-contained calibration method without external tools or sensors. To demonstrate the validity of our approach, we learn two challenging fine manipulation tasks: Rolling a marble and a bolt between two fingers. We show in simulation experiments that tactile feedback is crucial for precise manipulation and reaching sub-taxel resolution of < 1 mm (despite a taxel spacing of 4 mm). Moreover, we demonstrate that all policies successfully transfer from the simulation to the real robotic hand.