Ground truth force distribution for learning-based tactile sensing: a finite element approach

TL;DR

This paper introduces a finite element model to generate ground truth data for 3D force distribution in tactile sensing, enabling machine learning models to estimate detailed contact forces from images in real-time.

Contribution

It presents a novel finite element approach for obtaining ground truth force distributions, facilitating the training of neural networks for detailed tactile force reconstruction.

Findings

Finite element model accurately predicts force distribution.

Neural network trained on ground truth images can estimate forces in real-time.

High agreement between model predictions and experimental measurements.

Abstract

Skin-like tactile sensors provide robots with rich feedback related to the force distribution applied to their soft surface. The complexity of interpreting raw tactile information has driven the use of machine learning algorithms to convert the sensory feedback to the quantities of interest. However, the lack of ground truth sources for the entire contact force distribution has mainly limited these techniques to the sole estimation of the total contact force and the contact center on the sensor's surface. The method presented in this article uses a finite element model to obtain ground truth data for the three-dimensional force distribution. The model is obtained with state-of-the-art material characterization methods and is evaluated in an indentation setup, where it shows high agreement with the measurements retrieved from a commercial force-torque sensor. The proposed technique is applied to a vision-based tactile sensor, which aims to reconstruct the contact force distribution purely from images. Thousands of images are matched to ground truth data and are used to train a neural network architecture, which is suitable for real-time predictions.