Online Estimation of Table-Top Grown Strawberry Mass in Field Conditions with Occlusions

TL;DR

This paper presents a vision-based system combining RGB-D sensing, deep learning, and geometric modeling to accurately estimate strawberry mass in field conditions with occlusions, enabling real-time, non-destructive yield assessment.

Contribution

It introduces a novel pipeline integrating YOLOv8-Seg, CycleGAN, and tilt correction for improved occlusion handling and mass estimation of strawberries in the field.

Findings

Achieved mean mass estimation errors of 8.11% and 10.47% for isolated and occluded strawberries.

CycleGAN outperformed LaMa in occlusion recovery, with higher PAR and IoU scores.

Demonstrated robustness of the method in complex, occlusion-rich field conditions.

Abstract

Accurate mass estimation of table-top grown strawberries under field conditions remains challenging due to frequent occlusions and pose variations. This study proposes a vision-based pipeline integrating RGB-D sensing and deep learning to enable non-destructive, real-time and online mass estimation. The method employed YOLOv8-Seg for instance segmentation, Cycle-consistent generative adversarial network (CycleGAN) for occluded region completion, and tilt-angle correction to refine frontal projection area calculations. A polynomial regression model then mapped the geometric features to mass. Experiments demonstrated mean mass estimation errors of 8.11% for isolated strawberries and 10.47% for occluded cases. CycleGAN outperformed large mask inpainting (LaMa) model in occlusion recovery, achieving superior pixel area ratios (PAR) (mean: 0.978 vs. 1.112) and higher intersection over union (IoU) scores (92.3% vs. 47.7% in the [0.9-1] range). This approach addresses critical limitations of traditional methods, offering a robust solution for automated harvesting and yield monitoring with complex occlusion patterns.