Transfer Learning from Synthetic In-vitro Soybean Pods Dataset for In-situ Segmentation of On-branch Soybean Pod

TL;DR

This paper introduces a transfer learning approach using synthetic in-vitro soybean pod images to improve in-situ segmentation of on-branch soybean pods, addressing data scarcity and complex plant architecture.

Contribution

The study presents a novel synthetic dataset generation method and a two-step transfer learning process that enhances soybean pod segmentation accuracy in real-world scenarios.

Findings

Achieved AP$_{50}$ of 0.80 on real soybean data, outperforming direct adaptation.

Synthetic data effectively simulates touching pods for training.

Two-step transfer learning improves segmentation over baseline methods.

Abstract

The mature soybean plants are of complex architecture with pods frequently touching each other, posing a challenge for in-situ segmentation of on-branch soybean pods. Deep learning-based methods can achieve accurate training and strong generalization capabilities, but it demands massive labeled data, which is often a limitation, especially for agricultural applications. As lacking the labeled data to train an in-situ segmentation model for on-branch soybean pods, we propose a transfer learning from synthetic in-vitro soybean pods. First, we present a novel automated image generation method to rapidly generate a synthetic in-vitro soybean pods dataset with plenty of annotated samples. The in-vitro soybean pods samples are overlapped to simulate the frequently physically touching of on-branch soybean pods. Then, we design a two-step transfer learning. In the first step, we finetune an instance segmentation network pretrained by a source domain (MS COCO dataset) with a synthetic target domain (in-vitro soybean pods dataset). In the second step, transferring from simulation to reality is performed by finetuning on a few real-world mature soybean plant samples. The experimental results show the effectiveness of the proposed two-step transfer learning method, such that AP$_{50}$ was 0.80 for the real-world mature soybean plant test dataset, which is higher than that of direct adaptation and its AP$_{50}$ was 0.77. Furthermore, the visualizations of in-situ segmentation results of on-branch soybean pods show that our method performs better than other methods, especially when soybean pods overlap densely.