Efficacy of Image Similarity as a Metric for Augmenting Small Dataset Retinal Image Segmentation

TL;DR

This study investigates how the Fréchet Inception Distance (FID) metric relates to the effectiveness of synthetic images generated by a GAN in improving retinal image segmentation, revealing that lower FID correlates with better augmentation performance.

Contribution

It demonstrates that FID can predict the usefulness of synthetic images for dataset augmentation and shows synthetic data can outperform standard methods in improving segmentation.

Findings

Lower FID between datasets leads to better segmentation improvements.

Synthetic data can be more effective than standard augmentation techniques.

Dissimilar datasets (high FID) do not significantly improve segmentation.

Abstract

Synthetic images are an option for augmenting limited medical imaging datasets to improve the performance of various machine learning models. A common metric for evaluating synthetic image quality is the Fr\'echet Inception Distance (FID) which measures the similarity of two image datasets. In this study we evaluate the relationship between this metric and the improvement which synthetic images, generated by a Progressively Growing Generative Adversarial Network (PGGAN), grant when augmenting Diabetes-related Macular Edema (DME) intraretinal fluid segmentation performed by a U-Net model with limited amounts of training data. We find that the behaviour of augmenting with standard and synthetic images agrees with previously conducted experiments. Additionally, we show that dissimilar (high FID) datasets do not improve segmentation significantly. As FID between the training and augmenting datasets decreases, the augmentation datasets are shown to contribute to significant and robust improvements in image segmentation. Finally, we find that there is significant evidence to suggest that synthetic and standard augmentations follow separate log-normal trends between FID and improvements in model performance, with synthetic data proving more effective than standard augmentation techniques. Our findings show that more similar datasets (lower FID) will be more effective at improving U-Net performance, however, the results also suggest that this improvement may only occur when images are sufficiently dissimilar.