Viewpoint Planning for Fruit Size and Position Estimation

TL;DR

This paper introduces a novel viewpoint planning method for estimating fruit size and position in agriculture, effectively handling occlusions and improving detection accuracy through adaptive sampling and octree-based analysis.

Contribution

The paper presents a new viewpoint planning approach that combines ROI-focused sampling with exploration, utilizing an octree structure and heuristic utility for improved fruit detection.

Findings

Outperforms non-ROI sampling methods in simulated scenarios.

Successfully estimates fruit size and position in real-world greenhouse tests.

Demonstrates applicability on a robotic system with RGB-D sensors.

Abstract

Modern agricultural applications require knowledge about the position and size of fruits on plants. However, occlusions from leaves typically make obtaining this information difficult. We present a novel viewpoint planning approach that builds up an octree of plants with labeled regions of interest (ROIs), i.e., fruits. Our method uses this octree to sample viewpoint candidates that increase the information around the fruit regions and evaluates them using a heuristic utility function that takes into account the expected information gain. Our system automatically switches between ROI targeted sampling and exploration sampling, which considers general frontier voxels, depending on the estimated utility. When the plants have been sufficiently covered with the RGB-D sensor, our system clusters the ROI voxels and estimates the position and size of the detected fruits. We evaluated our approach in simulated scenarios and compared the resulting fruit estimations with the ground truth. The results demonstrate that our combined approach outperforms a sampling method that does not explicitly consider the ROIs to generate viewpoints in terms of the number of discovered ROI cells. Furthermore, we show the real-world applicability by testing our framework on a robotic arm equipped with an RGB-D camera installed on an automated pipe-rail trolley in a capsicum glasshouse.