SynMatch: Rethinking Consistency in Medical Image Segmentation with Sparse Annotations

TL;DR

SynMatch introduces a novel image synthesis framework that enhances medical image segmentation with sparse annotations by generating consistent image-label pairs, outperforming existing pseudo supervision methods especially in extremely limited annotation scenarios.

Contribution

It proposes a new framework that synthesizes images matching pseudo labels without extra training parameters, improving segmentation performance under limited annotations.

Findings

Outperforms recent pseudo supervision methods by up to 29.71% in polyp segmentation.

Achieves superior results across semi-, weakly-, and barely-supervised learning settings.

Demonstrates effectiveness especially in the most challenging annotation-limited BSL setting.

Abstract

Label scarcity remains a major challenge in deep learning-based medical image segmentation. Recent studies use strong-weak pseudo supervision to leverage unlabeled data. However, performance is often hindered by inconsistencies between pseudo labels and their corresponding unlabeled images. In this work, we propose \textbf{SynMatch}, a novel framework that sidesteps the need for improving pseudo labels by synthesizing images to match them instead. Specifically, SynMatch synthesizes images using texture and shape features extracted from the same segmentation model that generates the corresponding pseudo labels for unlabeled images. This design enables the generation of highly consistent synthesized-image-pseudo-label pairs without requiring any training parameters for image synthesis. We extensively evaluate SynMatch across diverse medical image segmentation tasks under semi-supervised learning (SSL), weakly-supervised learning (WSL), and barely-supervised learning (BSL) settings with increasingly limited annotations. The results demonstrate that SynMatch achieves superior performance, especially in the most challenging BSL setting. For example, it outperforms the recent strong-weak pseudo supervision-based method by 29.71\% and 10.05\% on the polyp segmentation task with 5\% and 10\% scribble annotations, respectively. The code will be released at https://github.com/Senyh/SynMatch.