ACROSS: A Deformation-Based Cross-Modal Representation for Robotic Tactile Perception

TL;DR

ACROSS is a framework that translates tactile sensor data between different modalities using deformation information, enabling reuse of datasets and cross-sensor data exchange in robotics.

Contribution

It introduces a novel deformation-based method for cross-modal tactile data translation, demonstrated by converting BioTac signals into DIGIT tactile images.

Findings

Successfully translated BioTac signals into DIGIT tactile images.

Enables reuse of tactile datasets across different sensor platforms.

Addresses the challenge of translating low-dimensional to high-dimensional tactile data.

Abstract

Tactile perception is essential for human interaction with the environment and is becoming increasingly crucial in robotics. Tactile sensors like the BioTac mimic human fingertips and provide detailed interaction data. Despite its utility in applications like slip detection and object identification, this sensor is now deprecated, making many valuable datasets obsolete. However, recreating similar datasets with newer sensor technologies is both tedious and time-consuming. Therefore, adapting these existing datasets for use with new setups and modalities is crucial. In response, we introduce ACROSS, a novel framework for translating data between tactile sensors by exploiting sensor deformation information. We demonstrate the approach by translating BioTac signals into the DIGIT sensor. Our framework consists of first converting the input signals into 3D deformation meshes. We then transition from the 3D deformation mesh of one sensor to the mesh of another, and finally convert the generated 3D deformation mesh into the corresponding output space. We demonstrate our approach to the most challenging problem of going from a low-dimensional tactile representation to a high-dimensional one. In particular, we transfer the tactile signals of a BioTac sensor to DIGIT tactile images. Our approach enables the continued use of valuable datasets and data exchange between groups with different setups.