Sparse 3D Perception for Rose Harvesting Robots: A Two-Stage Approach Bridging Simulation and Real-World Applications

TL;DR

This paper introduces a two-stage 3D perception pipeline for rose-harvesting robots, combining synthetic data generation, 2D detection, and depth estimation to enable accurate localization in real-world agricultural settings.

Contribution

It presents a novel sparse 3D localization method using synthetic data and deep learning, bridging the gap between simulation and real-world applications in agricultural robotics.

Findings

Achieved 95.6% F1 score in synthetic 2D detection

Attained 74.4% F1 score in real-world detection

Depth estimation error of 3% at 2 meters on synthetic data

Abstract

The global demand for medicinal plants, such as Damask roses, has surged with population growth, yet labor-intensive harvesting remains a bottleneck for scalability. To address this, we propose a novel 3D perception pipeline tailored for flower-harvesting robots, focusing on sparse 3D localization of rose centers. Our two-stage algorithm first performs 2D point-based detection on stereo images, followed by depth estimation using a lightweight deep neural network. To overcome the challenge of scarce real-world labeled data, we introduce a photorealistic synthetic dataset generated via Blender, simulating a dynamic rose farm environment with precise 3D annotations. This approach minimizes manual labeling costs while enabling robust model training. We evaluate two depth estimation paradigms: a traditional triangulation-based method and our proposed deep learning framework. Results demonstrate the superiority of our method, achieving an F1 score of 95.6% (synthetic) and 74.4% (real) in 2D detection, with a depth estimation error of 3% at a 2-meter range on synthetic data. The pipeline is optimized for computational efficiency, ensuring compatibility with resource-constrained robotic systems. By bridging the domain gap between synthetic and real-world data, this work advances agricultural automation for specialty crops, offering a scalable solution for precision harvesting.