Single-Shot 6DoF Pose and 3D Size Estimation for Robotic Strawberry Harvesting

TL;DR

This paper presents a deep-learning method for accurately estimating the 6DoF pose and 3D size of strawberries to improve robotic harvesting, trained on synthetic data and validated on real-world scenarios with high speed and occlusion robustness.

Contribution

The study introduces a novel deep-learning approach trained on synthetic data for simultaneous 6DoF pose and size estimation of strawberries, enhancing robotic harvesting efficiency.

Findings

Achieved 84.77% AP on synthetic dataset

Demonstrated effective real-world performance despite synthetic training

Reaches up to 60 FPS inference speed

Abstract

In this study, we introduce a deep-learning approach for determining both the 6DoF pose and 3D size of strawberries, aiming to significantly augment robotic harvesting efficiency. Our model was trained on a synthetic strawberry dataset, which is automatically generated within the Ignition Gazebo simulator, with a specific focus on the inherent symmetry exhibited by strawberries. By leveraging domain randomization techniques, the model demonstrated exceptional performance, achieving an 84.77\% average precision (AP) of 3D Intersection over Union (IoU) scores on the simulated dataset. Empirical evaluations, conducted by testing our model on real-world datasets, underscored the model's viability for real-world strawberry harvesting scenarios, even though its training was based on synthetic data. The model also exhibited robust occlusion handling abilities, maintaining accurate detection capabilities even when strawberries were obscured by other strawberries or foliage. Additionally, the model showcased remarkably swift inference speeds, reaching up to 60 frames per second (FPS).