FL-MedSegBench: A Comprehensive Benchmark for Federated Learning on Medical Image Segmentation

TL;DR

This paper introduces FL-MedSegBench, a comprehensive benchmark for evaluating federated learning methods on diverse medical image segmentation tasks, providing insights into method performance, fairness, communication efficiency, and generalization.

Contribution

It presents the first extensive benchmark for federated learning in medical image segmentation, systematically evaluating multiple methods across various clinical scenarios.

Findings

Personalized FL methods like FedBN outperform generic approaches.

No single method is best across all datasets.

Normalization-based personalization methods are robust to reduced communication.

Abstract

Federated learning (FL) offers a privacy-preserving paradigm for collaborative medical image analysis without sharing raw data. However, the absence of standardized benchmarks for medical image segmentation hinders fair and comprehensive evaluation of FL methods. To address this gap, we introduce FL-MedSegBench, the first comprehensive benchmark for federated learning on medical image segmentation. Our benchmark encompasses nine segmentation tasks across ten imaging modalities, covering both 2D and 3D formats with realistic clinical heterogeneity. We systematically evaluate eight generic FL (gFL) and five personalized FL (pFL) methods across multiple dimensions: segmentation accuracy, fairness, communication efficiency, convergence behavior, and generalization to unseen domains. Extensive experiments reveal several key insights: (i) pFL methods, particularly those with client-specific batch normalization (\textit{e.g.}, FedBN), consistently outperform generic approaches; (ii) No single method universally dominates, with performance being dataset-dependent; (iii) Communication frequency analysis shows normalization-based personalization methods exhibit remarkable robustness to reduced communication frequency; (iv) Fairness evaluation identifies methods like Ditto and FedRDN that protect underperforming clients; (v) A method's generalization to unseen domains is strongly tied to its ability to perform well across participating clients. We will release an open-source toolkit to foster reproducible research and accelerate clinically applicable FL solutions, providing empirically grounded guidelines for real-world clinical deployment. The source code is available at https://github.com/meiluzhu/FL-MedSegBench.