ControlTac: Force- and Position-Controlled Tactile Data Augmentation with a Single Reference Image

TL;DR

ControlTac is a novel framework that generates realistic, varied tactile images from a single reference image using contact force and position, enhancing tactile data augmentation for robotic applications.

Contribution

It introduces a two-stage controllable method for generating tactile images conditioned on physical priors, improving data realism and transferability.

Findings

Effective augmentation of tactile datasets demonstrated in three downstream tasks.

Generated tactile images are physically plausible and diverse.

Real-world experiments confirm practical utility.

Abstract

Vision-based tactile sensing has been widely used in perception, reconstruction, and robotic manipulation. However, collecting large-scale tactile data remains costly due to the localized nature of sensor-object interactions and inconsistencies across sensor instances. Existing approaches to scaling tactile data, such as simulation and free-form tactile generation, often suffer from unrealistic output and poor transferability to downstream tasks. To address this, we propose ControlTac, a two-stage controllable framework that generates realistic tactile images conditioned on a single reference tactile image, contact force, and contact position. With those physical priors as control input, ControlTac generates physically plausible and varied tactile images that can be used for effective data augmentation. Through experiments on three downstream tasks, we demonstrate that ControlTac can effectively augment tactile datasets and lead to consistent gains. Our three real-world experiments further validate the practical utility of our approach. Project page: https://dongyuluo.github.io/controltac.