Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections

TL;DR

This paper presents a fast, physics-based simulation model of a tactile sensor, combined with learned latent representations, enabling accurate transfer from simulation to real-world tactile sensing for robotic manipulation.

Contribution

It introduces an efficient FEM simulation of the BioTac sensor and a novel method for sim-to-real transfer using self-supervised latent learning and cross-modal projections.

Findings

Simulation speed increased by 75x

Accurate synthesis of real-world tactile signals

Effective transfer of tactile representations from simulation to real sensors

Abstract

Tactile sensing is critical for robotic grasping and manipulation of objects under visual occlusion. However, in contrast to simulations of robot arms and cameras, current simulations of tactile sensors have limited accuracy, speed, and utility. In this work, we develop an efficient 3D finite element method (FEM) model of the SynTouch BioTac sensor using an open-access, GPU-based robotics simulator. Our simulations closely reproduce results from an experimentally-validated model in an industry-standard, CPU-based simulator, but at 75x the speed. We then learn latent representations for simulated BioTac deformations and real-world electrical output through self-supervision, as well as projections between the latent spaces using a small supervised dataset. Using these learned latent projections, we accurately synthesize real-world BioTac electrical output and estimate contact patches, both for unseen contact interactions. This work contributes an efficient, freely-accessible FEM model of the BioTac and comprises one of the first efforts to combine self-supervision, cross-modal transfer, and sim-to-real transfer for tactile sensors.