Quantitative Hardness Assessment with Vision-based Tactile Sensing for Fruit Classification and Grasping

TL;DR

This paper introduces a vision-based tactile sensing framework for rapid, non-destructive fruit hardness assessment, improving robotic grasping and classification across various fruit types.

Contribution

It presents a novel, universal method for estimating fruit hardness using vision-based tactile sensors, enabling real-time, adaptive robotic handling in agriculture.

Findings

Effective normal force estimation from tactile images

Robust hardness assessment across multiple fruit types

Enhanced robotic grasping with reduced fruit damage

Abstract

Accurate estimation of fruit hardness is essential for automated classification and handling systems, particularly in determining fruit variety, assessing ripeness, and ensuring proper harvesting force. This study presents an innovative framework for quantitative hardness assessment utilizing vision-based tactile sensing, tailored explicitly for robotic applications in agriculture. The proposed methodology derives normal force estimation from a vision-based tactile sensor, and, based on the dynamics of this normal force, calculates the hardness. This approach offers a rapid, non-destructive evaluation through single-contact interaction. The integration of this framework into robotic systems enhances real-time adaptability of grasping forces, thereby reducing the likelihood of fruit damage. Moreover, the general applicability of this approach, through a universal criterion based on average normal force dynamics, ensures its effectiveness across a wide variety of fruit types and sizes. Extensive experimental validation conducted across different fruit types and ripeness-tracking studies demonstrates the efficacy and robustness of the framework, marking a significant advancement in the domain of automated fruit handling.