Improved Part Segmentation Performance by Optimising Realism of Synthetic Images using Cycle Generative Adversarial Networks

TL;DR

This study enhances plant part segmentation by translating synthetic images into a more realistic domain using cycle GANs, leading to significant improvements in CNN performance with less reliance on manual annotations.

Contribution

The paper introduces a cycle GAN-based method to improve synthetic image realism, boosting segmentation accuracy and reducing dependence on manual data annotation.

Findings

Translated synthetic images increased mean color correlation from 0.62 to 0.90.

Maximum IOU of 0.52 achieved with translated images and fine-tuning.

Training solely on translated synthetic images improved IOU by 55% over previous synthetic-only methods.

Abstract

In this paper we report on improved part segmentation performance using convolutional neural networks to reduce the dependency on the large amount of manually annotated empirical images. This was achieved by optimising the visual realism of synthetic agricultural images.In Part I, a cycle consistent generative adversarial network was applied to synthetic and empirical images with the objective to generate more realistic synthetic images by translating them to the empirical domain. We first hypothesise and confirm that plant part image features such as color and texture become more similar to the empirical domain after translation of the synthetic images.Results confirm this with an improved mean color distribution correlation with the empirical data prior of 0.62 and post translation of 0.90. Furthermore, the mean image features of contrast, homogeneity, energy and entropy moved closer to the empirical mean, post translation. In Part II, 7 experiments were performed using convolutional neural networks with different combinations of synthetic, synthetic translated to empirical and empirical images. We hypothesised that the translated images can be used for (i) improved learning of empirical images, and (ii) that learning without any fine-tuning with empirical images is improved by bootstrapping with translated images over bootstrapping with synthetic images. Results confirm our second and third hypotheses. First a maximum intersection-over-union performance was achieved of 0.52 when bootstrapping with translated images and fine-tuning with empirical images; an 8% increase compared to only using synthetic images. Second, training without any empirical fine-tuning resulted in an average IOU of 0.31; a 55% performance increase over previous methods that only used synthetic images.

Figures

Full text

See pages 1-last of contents.pdf