Geometry-Aware Fruit Grasping Estimation for Robotic Harvesting in Orchards

TL;DR

This paper introduces a geometry-aware neural network, A3N, for end-to-end fruit segmentation and grasping pose estimation in complex orchard environments, enhancing robotic harvesting accuracy and efficiency.

Contribution

The study presents a novel geometry-aware network and a global-to-local scanning strategy for improved fruit recognition and grasping in natural orchard settings.

Findings

A3N achieves 0.873 instance segmentation accuracy.

Robotic system attains 70-85% success rate in field harvesting.

Average grasping estimation accuracy is 0.61 cm and 4.8°.

Abstract

Field robotic harvesting is a promising technique in recent development of agricultural industry. It is vital for robots to recognise and localise fruits before the harvesting in natural orchards. However, the workspace of harvesting robots in orchards is complex: many fruits are occluded by branches and leaves. It is important to estimate a proper grasping pose for each fruit before performing the manipulation. In this study, a geometry-aware network, A3N, is proposed to perform end-to-end instance segmentation and grasping estimation using both color and geometry sensory data from a RGB-D camera. Besides, workspace geometry modelling is applied to assist the robotic manipulation. Moreover, we implement a global-to-local scanning strategy, which enables robots to accurately recognise and retrieve fruits in field environments with two consumer-level RGB-D cameras. We also evaluate the accuracy and robustness of proposed network comprehensively in experiments. The experimental results show that A3N achieves 0.873 on instance segmentation accuracy, with an average computation time of 35 ms. The average accuracy of grasping estimation is 0.61 cm and 4.8$^{\circ}$ in centre and orientation, respectively. Overall, the robotic system that utilizes the global-to-local scanning and A3N, achieves success rate of harvesting ranging from 70\% - 85\% in field harvesting experiments.