FOTS: A Fast Optical Tactile Simulator for Sim2Real Learning of Tactile-motor Robot Manipulation Skills

TL;DR

FOTS is a fast and efficient optical tactile simulator that enables zero-shot Sim2Real learning for tactile-motor robot manipulation, outperforming previous methods in speed and quality.

Contribution

We introduce FOTS, a novel optical tactile simulator that combines MLP mapping and planar shadow generation for high-speed, realistic tactile image and marker motion simulation.

Findings

FOTS achieves 28.6 fps for image simulation and 326.1 fps for marker motion on a CPU.

FOTS outperforms existing methods in image quality and rendering speed.

The simulator enables effective zero-shot Sim2Real transfer in tactile-motor tasks.

Abstract

Simulation is a widely used tool in robotics to reduce hardware consumption and gather large-scale data. Despite previous efforts to simulate optical tactile sensors, there remain challenges in efficiently synthesizing images and replicating marker motion under different contact loads. In this work, we propose a fast optical tactile simulator, named FOTS, for simulating optical tactile sensors. We utilize multi-layer perceptron mapping and planar shadow generation to simulate the optical response, while employing marker distribution approximation to simulate the motion of surface markers caused by the elastomer deformation. Experimental results demonstrate that FOTS outperforms other methods in terms of image generation quality and rendering speed, achieving 28.6 fps for optical simulation and 326.1 fps for marker motion simulation on a single CPU without GPU acceleration. In addition, we integrate the FOTS simulation model with physical engines like MuJoCo, and the peg-in-hole task demonstrates the effectiveness of our method in achieving zero-shot Sim2Real learning of tactile-motor robot manipulation skills. Our code is available at https://github.com/Rancho-zhao/FOTS.