Learn from Incomplete Tactile Data: Tactile Representation Learning with Masked Autoencoders

TL;DR

This paper introduces TacMAE, a masked autoencoder-based method for learning tactile representations from incomplete data, significantly improving tactile recognition accuracy and transferability in robotic perception tasks.

Contribution

The paper presents a novel masked autoencoder framework for tactile data, enabling effective learning from incomplete signals and enhancing tactile perception performance.

Findings

Achieves 71.4% accuracy in zero-shot tactile recognition

Improves recognition accuracy by 8.2% after fine-tuning

Demonstrates effective knowledge transfer on YCB objects

Abstract

The missing signal caused by the objects being occluded or an unstable sensor is a common challenge during data collection. Such missing signals will adversely affect the results obtained from the data, and this issue is observed more frequently in robotic tactile perception. In tactile perception, due to the limited working space and the dynamic environment, the contact between the tactile sensor and the object is frequently insufficient and unstable, which causes the partial loss of signals, thus leading to incomplete tactile data. The tactile data will therefore contain fewer tactile cues with low information density. In this paper, we propose a tactile representation learning method, named TacMAE, based on Masked Autoencoder to address the problem of incomplete tactile data in tactile perception. In our framework, a portion of the tactile image is masked out to simulate the missing contact region. By reconstructing the missing signals in the tactile image, the trained model can achieve a high-level understanding of surface geometry and tactile properties from limited tactile cues. The experimental results of tactile texture recognition show that our proposed TacMAE can achieve a high recognition accuracy of 71.4% in the zero-shot transfer and 85.8% after fine-tuning, which are 15.2% and 8.2% higher than the results without using masked modeling. The extensive experiments on YCB objects demonstrate the knowledge transferability of our proposed method and the potential to improve efficiency in tactile exploration.