DSFedMed: Dual-Scale Federated Medical Image Segmentation via Mutual Distillation Between Foundation and Lightweight Models

TL;DR

DSFedMed introduces a dual-scale federated learning framework that uses mutual knowledge distillation between a foundation model and lightweight client models, significantly improving medical image segmentation accuracy while greatly reducing communication and inference costs.

Contribution

The paper proposes a novel dual-scale federated framework with mutual distillation, high-quality synthetic data, and learnability-guided sample selection for efficient medical image segmentation.

Findings

Achieves 2% improvement in Dice score on average.

Reduces communication costs and inference time by nearly 90%.

Demonstrates scalability and efficiency in resource-limited settings.

Abstract

Foundation Models (FMs) have demonstrated strong generalization across diverse vision tasks. However, their deployment in federated settings is hindered by high computational demands, substantial communication overhead, and significant inference costs. We propose DSFedMed, a dual-scale federated framework that enables mutual knowledge distillation between a centralized foundation model and lightweight client models for medical image segmentation. To support knowledge distillation, a set of high-quality medical images is generated to replace real public datasets, and a learnability-guided sample selection strategy is proposed to enhance efficiency and effectiveness in dual-scale distillation. This mutual distillation enables the foundation model to transfer general knowledge to lightweight clients, while also incorporating client-specific insights to refine the foundation model. Evaluations on five medical imaging segmentation datasets show that DSFedMed achieves an average 2 percent improvement in Dice score while reducing communication costs and inference time by nearly 90 percent compared to existing federated foundation model baselines. These results demonstrate significant efficiency gains and scalability for resource-limited federated deployments.