Synthesizing brain tumor images and annotations by combining progressive growing GAN and SPADE

TL;DR

This paper explores combining two GANs to generate realistic brain tumor images and labels, aiming to augment training data for segmentation networks, with modest improvements observed.

Contribution

It introduces a novel method combining a noise-to-image GAN and an image-to-image GAN to synthesize labeled brain tumor images for data augmentation.

Findings

Synthesized images appear realistic

Adding synthetic images slightly improves segmentation

Method demonstrates potential for data augmentation

Abstract

Training segmentation networks requires large annotated datasets, but manual annotation is time consuming and costly. We here investigate if the combination of a noise-to-image GAN and an image-to-image GAN can be used to synthesize realistic brain tumor images as well as the corresponding tumor annotations (labels), to substantially increase the number of training images. The noise-to-image GAN is used to synthesize new label images, while the image-to-image GAN generates the corresponding MR image from the label image. Our results indicate that the two GANs can synthesize label images and MR images that look realistic, and that adding synthetic images improves the segmentation performance, although the effect is small.