Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

TL;DR

This paper introduces a novel 3D multi-modal medical image segmentation method using GANs that effectively leverages limited labeled data, extending adversarial learning to complex 3D multi-modal volumes, and demonstrates significant performance improvements.

Contribution

It extends adversarial learning to 3D multi-modal segmentation with few labeled examples, incorporating feature matching for better semi-supervised performance.

Findings

Significant performance gains over fully-supervised methods.

Effective extension of GAN-based semi-supervised learning to 3D multi-modal data.

Feature matching improves segmentation accuracy.

Abstract

We address the problem of segmenting 3D multi-modal medical images in scenarios where very few labeled examples are available for training. Leveraging the recent success of adversarial learning for semi-supervised segmentation, we propose a novel method based on Generative Adversarial Networks (GANs) to train a segmentation model with both labeled and unlabeled images. The proposed method prevents over-fitting by learning to discriminate between true and fake patches obtained by a generator network. Our work extends current adversarial learning approaches, which focus on 2D single-modality images, to the more challenging context of 3D volumes of multiple modalities. The proposed method is evaluated on the problem of segmenting brain MRI from the iSEG-2017 and MRBrainS 2013 datasets. Significant performance improvement is reported, compared to state-of-art segmentation networks trained in a fully-supervised manner. In addition, our work presents a comprehensive analysis of different GAN architectures for semi-supervised segmentation, showing recent techniques like feature matching to yield a higher performance than conventional adversarial training approaches. Our code is publicly available at https://github.com/arnab39/FewShot_GAN-Unet3D