Annotation-Efficient Task Guidance for Medical Segment Anything

TL;DR

SAM-Mix introduces an annotation-efficient multitask learning framework that significantly improves medical image segmentation accuracy using minimal labeled data, demonstrating strong results on liver segmentation from CT scans.

Contribution

It presents a novel multitask learning approach combining class activation maps with SAM for annotation-efficient medical segmentation.

Findings

Achieves 5.1% Dice improvement with 90% fewer training epochs and only 0.04% labeled data.

Yields 25.4% Dice improvement on cross-domain segmentation.

Effective in simultaneous classification and segmentation tasks.

Abstract

Medical image segmentation is a key task in the imaging workflow, influencing many image-based decisions. Traditional, fully-supervised segmentation models rely on large amounts of labeled training data, typically obtained through manual annotation, which can be an expensive, time-consuming, and error-prone process. This signals a need for accurate, automatic, and annotation-efficient methods of training these models. We propose SAM-Mix, a novel multitask learning framework for medical image segmentation that uses class activation maps produced by an auxiliary classifier to guide the predictions of the semi-supervised segmentation branch, which is based on the SAM framework. Experimental evaluations on the public LiTS dataset confirm the effectiveness of SAM-Mix for simultaneous classification and segmentation of the liver from abdominal computed tomography (CT) scans. When trained for 90% fewer epochs on only 50 labeled 2D slices, representing just 0.04% of the available labeled training data, SAM-Mix achieves a Dice improvement of 5.1% over the best baseline model. The generalization results for SAM-Mix are even more impressive, with the same model configuration yielding a 25.4% Dice improvement on a cross-domain segmentation task. Our code is available at https://github.com/tbwa233/SAM-Mix.