Tactile Image-to-Image Disentanglement of Contact Geometry from Motion-Induced Shear

TL;DR

This paper introduces a supervised deep learning model that disentangles contact geometry from shear-induced deformation in tactile images, enabling accurate shape reconstruction and pose estimation for robotic touch.

Contribution

A novel neural network approach that separates shear effects from contact geometry in tactile images, improving shape reconstruction and robotic manipulation.

Findings

Successfully reconstructs unsheared tactile images from sheared data.

Enables accurate contact geometry and pose estimation.

Facilitates full object shape reconstruction using tactile sensing.

Abstract

Robotic touch, particularly when using soft optical tactile sensors, suffers from distortion caused by motion-dependent shear. The manner in which the sensor contacts a stimulus is entangled with the tactile information about the geometry of the stimulus. In this work, we propose a supervised convolutional deep neural network model that learns to disentangle, in the latent space, the components of sensor deformations caused by contact geometry from those due to sliding-induced shear. The approach is validated by reconstructing unsheared tactile images from sheared images and showing they match unsheared tactile images collected with no sliding motion. In addition, the unsheared tactile images give a faithful reconstruction of the contact geometry that is not possible from the sheared data, and robust estimation of the contact pose that can be used for servo control sliding around various 2D shapes. Finally, the contact geometry reconstruction in conjunction with servo control sliding were used for faithful full object reconstruction of various 2D shapes. The methods have broad applicability to deep learning models for robots with a shear-sensitive sense of touch.