Toward Artificial Palpation: Representation Learning of Touch on Soft Bodies

TL;DR

This paper proposes a self-supervised learning approach to develop an artificial palpation system that learns tactile representations from soft objects, enabling tasks like imaging and change detection, validated through simulation and real-world MRI data.

Contribution

It introduces a novel encoder-decoder framework for tactile representation learning from palpation sequences, advancing beyond force maps to capture complex tactile patterns.

Findings

Learned tactile representations enable imaging.

Effective change detection on soft objects.

Validated with real-world MRI data.

Abstract

Palpation, the use of touch in medical examination, is almost exclusively performed by humans. We investigate a proof of concept for an artificial palpation method based on self-supervised learning. Our key idea is that an encoder-decoder framework can learn a $\textit{representation}$ from a sequence of tactile measurements that contains all the relevant information about the palpated object. We conjecture that such a representation can be used for downstream tasks such as tactile imaging and change detection. With enough training data, it should capture intricate patterns in the tactile measurements that go beyond a simple map of forces -- the current state of the art. To validate our approach, we both develop a simulation environment and collect a real-world dataset of soft objects and corresponding ground truth images obtained by magnetic resonance imaging (MRI). We collect palpation sequences using a robot equipped with a tactile sensor, and train a model that predicts sensory readings at different positions on the object. We investigate the representation learned in this process, and demonstrate its use in imaging and change detection.