Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation

TL;DR

This paper introduces UMCT, a framework that leverages multi-view co-training with uncertainty estimation to improve semi-supervised and domain-adaptive 3D medical image segmentation, achieving state-of-the-art results.

Contribution

The paper presents a novel uncertainty-aware multi-view co-training framework that unifies semi-supervised learning and domain adaptation for volumetric medical image segmentation.

Findings

State-of-the-art performance on NIH pancreas segmentation dataset

Effective domain adaptation to pathological organs from Decathlon datasets

Robustness in scenarios with inaccessible labeled source data

Abstract

Although having achieved great success in medical image segmentation, deep learning-based approaches usually require large amounts of well-annotated data, which can be extremely expensive in the field of medical image analysis. Unlabeled data, on the other hand, is much easier to acquire. Semi-supervised learning and unsupervised domain adaptation both take the advantage of unlabeled data, and they are closely related to each other. In this paper, we propose uncertainty-aware multi-view co-training (UMCT), a unified framework that addresses these two tasks for volumetric medical image segmentation. Our framework is capable of efficiently utilizing unlabeled data for better performance. We firstly rotate and permute the 3D volumes into multiple views and train a 3D deep network on each view. We then apply co-training by enforcing multi-view consistency on unlabeled data, where an uncertainty estimation of each view is utilized to achieve accurate labeling. Experiments on the NIH pancreas segmentation dataset and a multi-organ segmentation dataset show state-of-the-art performance of the proposed framework on semi-supervised medical image segmentation. Under unsupervised domain adaptation settings, we validate the effectiveness of this work by adapting our multi-organ segmentation model to two pathological organs from the Medical Segmentation Decathlon Datasets. Additionally, we show that our UMCT-DA model can even effectively handle the challenging situation where labeled source data is inaccessible, demonstrating strong potentials for real-world applications.