Learning In-Hand Translation Using Tactile Skin With Shear and Normal Force Sensing

TL;DR

This paper presents a tactile skin sensor model enabling zero-shot sim-to-real transfer for in-hand translation tasks, leveraging shear and normal forces to improve dexterous manipulation in real-world scenarios.

Contribution

The authors introduce a novel tactile sensor model and RL policy that utilize shear and normal forces for dexterous in-hand translation, achieving effective sim-to-real transfer.

Findings

Tactile policies outperform baselines using only shear or normal forces.

Zero-shot sim-to-real transfer is achieved with the proposed sensor model.

Tactile sensing enhances policy adaptation to unseen objects and orientations.

Abstract

Recent progress in reinforcement learning (RL) and tactile sensing has significantly advanced dexterous manipulation. However, these methods often utilize simplified tactile signals due to the gap between tactile simulation and the real world. We introduce a sensor model for tactile skin that enables zero-shot sim-to-real transfer of ternary shear and binary normal forces. Using this model, we develop an RL policy that leverages sliding contact for dexterous in-hand translation. We conduct extensive real-world experiments to assess how tactile sensing facilitates policy adaptation to various unseen object properties and robot hand orientations. We demonstrate that our 3-axis tactile policies consistently outperform baselines that use only shear forces, only normal forces, or only proprioception. Website: https://jessicayin.github.io/tactile-skin-rl/