Rethinking Data Perturbation and Model Stabilization for Semi-supervised Medical Image Segmentation

TL;DR

This paper introduces DPMS, a simple yet effective semi-supervised medical image segmentation method that emphasizes data perturbation and model stabilization, achieving state-of-the-art results by generating prediction disagreement on unlabeled data.

Contribution

The paper rethinks core challenges in semi-supervised segmentation and proposes DPMS, a straightforward approach leveraging data augmentation and stabilization techniques for improved performance.

Findings

DPMS achieves state-of-the-art results on ACDC and LA datasets.

Strong data augmentation significantly increases prediction disagreement.

Stabilization strategies improve training effectiveness on unlabeled data.

Abstract

Studies on semi-supervised medical image segmentation (SSMIS) have seen fast progress recently. Due to the limited labelled data, SSMIS methods mainly focus on effectively leveraging unlabeled data to enhance the segmentation performance. However, despite their promising performance, current state-of-the-art methods often prioritize integrating complex techniques and loss terms rather than addressing the core challenges of semi-supervised scenarios directly. We argue that the key to SSMIS lies in generating substantial and appropriate prediction disagreement on unlabeled data. To this end, we emphasize the crutiality of data perturbation and model stabilization in semi-supervised segmentation, and propose a simple yet effective approach to boost SSMIS performance significantly, dubbed DPMS. Specifically, we first revisit SSMIS from three distinct perspectives: the data, the model, and the loss, and conduct a comprehensive study of corresponding strategies to examine their effectiveness. Based on these examinations, we then propose DPMS, which adopts a plain teacher-student framework with a standard supervised loss and unsupervised consistency loss. To produce appropriate prediction disagreements, DPMS perturbs the unlabeled data via strong augmentations to enlarge prediction disagreements considerably. On the other hand, using EMA teacher when strong augmentation is applied does not necessarily improve performance. DPMS further utilizes a forwarding-twice and momentum updating strategies for normalization statistics to stabilize the training on unlabeled data effectively. Despite its simplicity, DPMS can obtain new state-of-the-art performance on the public 2D ACDC and 3D LA datasets across various semi-supervised settings, e.g. obtaining a remarkable 22.62% improvement against previous SOTA on ACDC with 5% labels.