GS-NBV: a Geometry-based, Semantics-aware Viewpoint Planning Algorithm for Avocado Harvesting under Occlusions

TL;DR

This paper introduces GS-NBV, a geometry-based, semantics-aware viewpoint planning algorithm designed to improve automated avocado harvesting by effectively handling occlusions and irregular shapes, achieving high success rates in simulations.

Contribution

The paper presents a novel viewpoint planning method that combines geometric and semantic information for better occlusion handling in automated fruit harvesting.

Findings

Achieved 100% success rate in simulation case studies.

Effectively handles occlusions with a geometry-based approach.

Outperforms two state-of-the-art algorithms in validation tests.

Abstract

Efficient identification of picking points is critical for automated fruit harvesting. Avocados present unique challenges owing to their irregular shape, weight, and less-structured growing environments, which require specific viewpoints for successful harvesting. We propose a geometry-based, semantics-aware viewpoint-planning algorithm to address these challenges. The planning process involves three key steps: viewpoint sampling, evaluation, and execution. Starting from a partially occluded view, the system first detects the fruit, then leverages geometric information to constrain the viewpoint search space to a 1D circle, and uniformly samples four points to balance the efficiency and exploration. A new picking score metric is introduced to evaluate the viewpoint suitability and guide the camera to the next-best view. We validate our method through simulation against two state-of-the-art algorithms. Results show a 100% success rate in two case studies with significant occlusions, demonstrating the efficiency and robustness of our approach. Our code is available at https://github.com/lineojcd/GSNBV