Synthetic Crop-Weed Image Generation and its Impact on Model Generalization

TL;DR

This paper introduces a Blender-based pipeline for generating diverse synthetic crop-weed images to improve semantic segmentation models for agricultural robots, addressing data scarcity and domain gap issues.

Contribution

It presents a novel procedural synthetic data generation method and benchmarks its effectiveness, showing improved model generalization across domains.

Findings

Synthetic data reduces annotation costs.

Models trained on synthetic data generalize well across domains.

The sim-to-real gap is minimized to 10% with the proposed approach.

Abstract

Precise semantic segmentation of crops and weeds is necessary for agricultural weeding robots. However, training deep learning models requires large annotated datasets, which are costly to obtain in real fields. Synthetic data can reduce this burden, but the gap between simulated and real images remains a challenge. In this paper, we present a pipeline for procedural generation of synthetic crop-weed images using Blender, producing annotated datasets under diverse conditions of plant growth, weed density, lighting, and camera angle. We benchmark several state-of-the-art segmentation models on synthetic and real datasets and analyze their cross-domain generalization. Our results show that training on synthetic images leads to a sim-to-real gap of 10%, surpassing previous state-of-the-art methods. Moreover, synthetic data demonstrates good generalization properties, outperforming real datasets in cross-domain scenarios. These findings highlight the potential of synthetic agricultural datasets and support hybrid strategies for more efficient model training.