Tactile Functasets: Neural Implicit Representations of Tactile Datasets

TL;DR

This paper introduces neural implicit functions to represent tactile sensor data compactly, improving efficiency, interpretability, and generalization, and demonstrates their effectiveness in in-hand object pose estimation.

Contribution

The paper presents a novel neural implicit representation for tactile datasets, enabling compact storage, probabilistic inference, and sensor-agnostic generalization.

Findings

Improved in-hand object pose estimation performance.

Compact and interpretable tactile data representations.

Facilitated generalization across different tactile sensors.

Abstract

Modern incarnations of tactile sensors produce high-dimensional raw sensory feedback such as images, making it challenging to efficiently store, process, and generalize across sensors. To address these concerns, we introduce a novel implicit function representation for tactile sensor feedback. Rather than directly using raw tactile images, we propose neural implicit functions trained to reconstruct the tactile dataset, producing compact representations that capture the underlying structure of the sensory inputs. These representations offer several advantages over their raw counterparts: they are compact, enable probabilistically interpretable inference, and facilitate generalization across different sensors. We demonstrate the efficacy of this representation on the downstream task of in-hand object pose estimation, achieving improved performance over image-based methods while simplifying downstream models. We release code, demos and datasets at https://www.mmintlab.com/tactile-functasets.