From Few to More: Scribble-based Medical Image Segmentation via Masked Context Modeling and Continuous Pseudo Labels

TL;DR

This paper introduces MaCo, a novel weakly supervised medical image segmentation model that uses masked context modeling and continuous pseudo labels to improve segmentation accuracy with sparse annotations.

Contribution

MaCo is the first to integrate Masked Context Modeling and continuous pseudo labels specifically for scribble-based medical image segmentation, addressing challenges of sparse annotations.

Findings

MaCo outperforms existing weakly supervised methods on three public datasets.

MaCo establishes new state-of-the-art results in scribble-based medical image segmentation.

The use of continuous pseudo labels improves the reliability of supervision from sparse annotations.

Abstract

Scribble-based weakly supervised segmentation methods have shown promising results in medical image segmentation, significantly reducing annotation costs. However, existing approaches often rely on auxiliary tasks to enforce semantic consistency and use hard pseudo labels for supervision, overlooking the unique challenges faced by models trained with sparse annotations. These models must predict pixel-wise segmentation maps from limited data, making it crucial to handle varying levels of annotation richness effectively. In this paper, we propose MaCo, a weakly supervised model designed for medical image segmentation, based on the principle of "from few to more." MaCo leverages Masked Context Modeling (MCM) and Continuous Pseudo Labels (CPL). MCM employs an attention-based masking strategy to perturb the input image, ensuring that the model's predictions align with those of the original image. CPL converts scribble annotations into continuous pixel-wise labels by applying an exponential decay function to distance maps, producing confidence maps that represent the likelihood of each pixel belonging to a specific category, rather than relying on hard pseudo labels. We evaluate MaCo on three public datasets, comparing it with other weakly supervised methods. Our results show that MaCo outperforms competing methods across all datasets, establishing a new record in weakly supervised medical image segmentation.