Advances in Compliance Detection: Novel Models Using Vision-Based Tactile Sensors

TL;DR

This paper introduces two advanced neural network models using vision-based tactile sensors to improve compliance detection accuracy across various applications.

Contribution

The paper presents novel LRCN and Transformer models that enhance compliance prediction accuracy using RGB tactile images from GelSight sensors.

Findings

Models outperform baseline methods in compliance prediction.

Transformer-based model achieves higher accuracy than LRCN.

Object hardness affects compliance estimation difficulty.

Abstract

Compliance is a critical parameter for describing objects in engineering, agriculture, and biomedical applications. Traditional compliance detection methods are limited by their lack of portability and scalability, rely on specialized, often expensive equipment, and are unsuitable for robotic applications. Moreover, existing neural network-based approaches using vision-based tactile sensors still suffer from insufficient prediction accuracy. In this paper, we propose two models based on Long-term Recurrent Convolutional Networks (LRCNs) and Transformer architectures that leverage RGB tactile images and other information captured by the vision-based sensor GelSight to predict compliance metrics accurately. We validate the performance of these models using multiple metrics and demonstrate their effectiveness in accurately estimating compliance. The proposed models exhibit significant performance improvement over the baseline. Additionally, we investigated the correlation between sensor compliance and object compliance estimation, which revealed that objects that are harder than the sensor are more challenging to estimate.