An analysis of sensor selection for fruit picking with suction-based grippers

TL;DR

This paper evaluates multimodal sensor sets integrated into a suction-based apple gripper, identifying minimal sensors and phase-dependent strategies for reliable pick success and failure detection in orchard conditions.

Contribution

It introduces a phase-dependent sensor evaluation and minimal sensor set identification for improved pick and slip detection in robotic fruit harvesting.

Findings

Random Forest and MLP classifiers achieve over 90% accuracy in pick success detection.

Random Forest predicts pick/slip events within 0.09 seconds of ground truth.

Phase-dependent sensing improves detection reliability in orchard environments.

Abstract

Robotic fruit harvesting often fails to reliably detect whether a fruit has been successfully picked, limiting efficiency and increasing crop damage. This problem is difficult due to compliant fruit and grippers, variable stem attachment, and occlusions in orchard environments. Prior work has explored vision-based perception and multi-sensor learning approaches for pick state estimation. However, minimal sensor sets and phase-dependent sensing strategies for accurate pick and slip detection remain largely unexplored. In this work, we design and evaluate a multimodal sensing suite integrated into a compliant suction-based apple gripper. Our approach is unique because it identifies which sensors are most informative at different phases of the pick, enabling predictive detection of failures before they occur. The contributions of this paper are a phase-dependent evaluation of multimodal sensors and the identification of minimal sensor sets for reliable pick state classification. Experiments in a real apple orchard show that Random Forest and Multilayer Perceptron classifiers detect successful picks and impending failures with over 90% accuracy, and Random Forest predicts pick/slip events within 0.09 s of human-annotated ground truth.