Learning Force Distribution Estimation for the GelSight Mini Optical Tactile Sensor Based on Finite Element Analysis

TL;DR

This paper presents a machine learning method using a U-net architecture to accurately estimate shear and normal force distributions from GelSight Mini tactile sensor images, enabling improved contact-rich manipulation in robotics.

Contribution

It introduces a novel approach combining finite element analysis with deep learning to predict force distributions from tactile sensor data, advancing contact sensing capabilities.

Findings

High accuracy in force distribution prediction

Good generalization across different indenters and sensors

Potential for real-time tactile force estimation

Abstract

Contact-rich manipulation remains a major challenge in robotics. Optical tactile sensors like GelSight Mini offer a low-cost solution for contact sensing by capturing soft-body deformations of the silicone gel. However, accurately inferring shear and normal force distributions from these gel deformations has yet to be fully addressed. In this work, we propose a machine learning approach using a U-net architecture to predict force distributions directly from the sensor's raw images. Our model, trained on force distributions inferred from \ac{fea}, demonstrates promising accuracy in predicting normal and shear force distributions for the commercially available GelSight Mini sensor. It also shows potential for generalization across indenters, sensors of the same type, and for enabling real-time application. The codebase, dataset and models are open-sourced and available at https://feats-ai.github.io .