NBV-SC: Next Best View Planning based on Shape Completion for Fruit Mapping and Reconstruction

TL;DR

This paper introduces a novel next best view planning method for fruit mapping that leverages shape completion to efficiently observe unscanned parts, improving reconstruction accuracy and reducing planning time.

Contribution

The approach explicitly uses predicted fruit shapes for targeted viewpoint planning and introduces viewpoint dissimilarity to optimize viewpoint sampling.

Findings

Improves fruit size estimation accuracy.

Enhances fruit reconstruction quality.

Reduces planning time compared to state-of-the-art methods.

Abstract

Active perception for fruit mapping and harvesting is a difficult task since occlusions occur frequently and the location as well as size of fruits change over time. State-of-the-art viewpoint planning approaches utilize computationally expensive ray casting operations to find good viewpoints aiming at maximizing information gain and covering the fruits in the scene. In this paper, we present a novel viewpoint planning approach that explicitly uses information about the predicted fruit shapes to compute targeted viewpoints that observe as yet unobserved parts of the fruits. Furthermore, we formulate the concept of viewpoint dissimilarity to reduce the sampling space for more efficient selection of useful, dissimilar viewpoints. Our simulation experiments with a UR5e arm equipped with an RGB-D sensor provide a quantitative demonstration of the efficacy of our iterative next best view planning method based on shape completion. In comparative experiments with a state-of-the-art viewpoint planner, we demonstrate improvement not only in the estimation of the fruit sizes, but also in their reconstruction, while significantly reducing the planning time. Finally, we show the viability of our approach for mapping sweet peppers plants with a real robotic system in a commercial glasshouse.