TACTO: A Fast, Flexible, and Open-source Simulator for High-Resolution Vision-based Tactile Sensors

TL;DR

TACTO is an open-source, high-speed simulator for vision-based tactile sensors, enabling realistic touch simulation for robotics research and machine learning applications, including grasp stability prediction and marble manipulation.

Contribution

We introduce TACTO, a novel simulator that accurately and efficiently models high-resolution vision-based tactile sensors for robotics and learning tasks.

Findings

Realistic high-resolution touch readings at hundreds of frames per second

Successful prediction of grasp stability from simulated touch data

Effective use of TACTO for Sim2Real transfer in manipulation tasks

Abstract

Simulators perform an important role in prototyping, debugging, and benchmarking new advances in robotics and learning for control. Although many physics engines exist, some aspects of the real world are harder than others to simulate. One of the aspects that have so far eluded accurate simulation is touch sensing. To address this gap, we present TACTO - a fast, flexible, and open-source simulator for vision-based tactile sensors. This simulator allows to render realistic high-resolution touch readings at hundreds of frames per second, and can be easily configured to simulate different vision-based tactile sensors, including DIGIT and OmniTact. In this paper, we detail the principles that drove the implementation of TACTO and how they are reflected in its architecture. We demonstrate TACTO on a perceptual task, by learning to predict grasp stability using touch from 1 million grasps, and on a marble manipulation control task. Moreover, we provide a proof-of-concept that TACTO can be successfully used for Sim2Real applications. We believe that TACTO is a step towards the widespread adoption of touch sensing in robotic applications, and to enable machine learning practitioners interested in multi-modal learning and control. TACTO is open-source at https://github.com/facebookresearch/tacto.