TensorTouch: Calibration of Tactile Sensors for High Resolution Stress Tensor and Deformation for Dexterous Manipulation

TL;DR

TensorTouch combines finite element analysis and deep learning to calibrate optical tactile sensors, enabling high-resolution stress and deformation measurements crucial for advanced robotic manipulation tasks.

Contribution

It introduces a novel framework that extracts detailed contact metrics from optical tactile sensors, improving interpretability and transferability for dexterous manipulation.

Findings

Achieves sub-millimeter position accuracy.

Demonstrates 90% success in string grasping task.

Supports large deformations for soft object manipulation.

Abstract

Advanced dexterous manipulation involving multiple simultaneous contacts across different surfaces, like pinching coins from ground or manipulating intertwined objects, remains challenging for robotic systems. Such tasks exceed the capabilities of vision and proprioception alone, requiring high-resolution tactile sensing with calibrated physical metrics. Raw optical tactile sensor images, while information-rich, lack interpretability and cross-sensor transferability, limiting their real-world utility. TensorTouch addresses this challenge by integrating finite element analysis with deep learning to extract comprehensive contact information from optical tactile sensors, including stress tensors, deformation fields, and force distributions at pixel-level resolution. The TensorTouch framework achieves sub-millimeter position accuracy and precise force estimation while supporting large sensor deformations crucial for manipulating soft objects. Experimental validation demonstrates 90% success in selectively grasping one of two strings based on detected motion, enabling new contact-rich manipulation capabilities previously inaccessible to robotic systems.