Investigating Sensors and Methods in Grasp State Classification in Agricultural Manipulation

TL;DR

This paper evaluates various sensors and machine learning models for classifying grasp states in agricultural harvesting, demonstrating high accuracy with a minimal sensor set and highlighting the importance of sensor selection for reliable feedback.

Contribution

It systematically compares sensors and models for grasp state classification, identifying an optimal minimal sensor combination and achieving high accuracy in real-world conditions.

Findings

Random Forest achieved 100% accuracy in lab-to-field tests.

IMU and tension sensors form an effective minimal sensor set.

The approach enables real-time corrective actions in harvesting.

Abstract

Effective and efficient agricultural manipulation and harvesting depend on accurately understanding the current state of the grasp. The agricultural environment presents unique challenges due to its complexity, clutter, and occlusion. Additionally, fruit is physically attached to the plant, requiring precise separation during harvesting. Selecting appropriate sensors and modeling techniques is critical for obtaining reliable feedback and correctly identifying grasp states. This work investigates a set of key sensors, namely inertial measurement units (IMUs), infrared (IR) reflectance, tension, tactile sensors, and RGB cameras, integrated into a compliant gripper to classify grasp states. We evaluate the individual contribution of each sensor and compare the performance of two widely used classification models: Random Forest and Long Short-Term Memory (LSTM) networks. Our results demonstrate that a Random Forest classifier, trained in a controlled lab environment and tested on real cherry tomato plants, achieved 100% accuracy in identifying slip, grasp failure, and successful picks, marking a substantial improvement over baseline performance. Furthermore, we identify a minimal viable sensor combination, namely IMU and tension sensors that effectively classifies grasp states. This classifier enables the planning of corrective actions based on real-time feedback, thereby enhancing the efficiency and reliability of fruit harvesting operations.