SAM-Fed: SAM-Guided Federated Semi-Supervised Learning for Medical Image Segmentation

TL;DR

SAM-Fed introduces a federated semi-supervised learning framework that uses a high-capacity foundation model to guide lightweight clients, improving medical image segmentation accuracy while addressing data privacy and resource constraints.

Contribution

The paper presents SAM-Fed, a novel approach that combines foundation model guidance with dual knowledge distillation for federated semi-supervised segmentation.

Findings

Outperforms state-of-the-art FSSL methods in skin lesion and polyp segmentation.

Effective in both homogeneous and heterogeneous client settings.

Enhances pseudo-label quality and stability across diverse models.

Abstract

Medical image segmentation is clinically important, yet data privacy and the cost of expert annotation limit the availability of labeled data. Federated semi-supervised learning (FSSL) offers a solution but faces two challenges: pseudo-label reliability depends on the strength of local models, and client devices often require compact or heterogeneous architectures due to limited computational resources. These constraints reduce the quality and stability of pseudo-labels, while large models, though more accurate, cannot be trained or used for routine inference on client devices. We propose SAM-Fed, a federated semi-supervised framework that leverages a high-capacity segmentation foundation model to guide lightweight clients during training. SAM-Fed combines dual knowledge distillation with an adaptive agreement mechanism to refine pixel-level supervision. Experiments on skin lesion and polyp segmentation across homogeneous and heterogeneous settings show that SAM-Fed consistently outperforms state-of-the-art FSSL methods.