HydroShear: Hydroelastic Shear Simulation for Tactile Sim-to-Real Reinforcement Learning

TL;DR

HydroShear introduces a hydroelastic tactile simulator that accurately models shear forces and contact interactions, enabling effective zero-shot transfer of tactile policies in robotic manipulation tasks.

Contribution

The paper presents HydroShear, a novel hydroelastic tactile simulator that models complex force interactions, improving sim-to-real transfer for contact-rich manipulation tasks.

Findings

HydroShear more accurately reproduces real tactile shear than existing methods.

Zero-shot policy transfer achieves 93% success rate across multiple tasks.

Outperforms tactile image-based and alternative shear simulation methods.

Abstract

In this paper, we address the problem of tactile sim-to-real policy transfer for contact-rich tasks. Existing methods primarily focus on vision-based sensors and emphasize image rendering quality while providing overly simplistic models of force and shear. Consequently, these models exhibit a large sim-to-real gap for many dexterous tasks. Here, we present HydroShear, a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions. HydroShear extends hydroelastic contact models using Signed Distance Functions (SDFs) to track the displacements of the on-surface points of an indenter during physical interaction with the sensor membrane. Our approach generates physics-based, computationally efficient force fields from arbitrary watertight geometries while remaining agnostic to the underlying physics engine. In experiments with GelSight Minis, HydroShear more faithfully reproduces real tactile shear compared to existing methods. This fidelity enables zero-shot sim-to-real transfer of reinforcement learning policies across four tasks: peg insertion, bin packing, book shelving for insertion, and drawer pulling for fine gripper control under slip. Our method achieves a 93% average success rate, outperforming policies trained on tactile images (34%) and alternative shear simulation methods (58%-61%).