Federated Learning with Partially Labeled Data: A Conditional Distillation Approach

TL;DR

This paper introduces ConDistFL, a federated learning framework with conditional distillation, designed to improve medical image segmentation from partially labeled data while preserving privacy and scalability.

Contribution

The paper presents a novel federated learning approach that effectively handles partial labels through conditional distillation, enhancing segmentation accuracy and generalizability in medical imaging.

Findings

Significantly improves segmentation accuracy across distributed datasets.

Maintains computational and communication efficiency.

Outperforms existing FL methods in out-of-federation tests.

Abstract

In medical imaging, developing generalized segmentation models that can handle multiple organs and lesions is crucial. However, the scarcity of fully annotated datasets and strict privacy regulations present significant barriers to data sharing. Federated Learning (FL) allows decentralized model training, but existing FL methods often struggle with partial labeling, leading to model divergence and catastrophic forgetting. We propose ConDistFL, a novel FL framework incorporating conditional distillation to address these challenges. ConDistFL enables effective learning from partially labeled datasets, significantly improving segmentation accuracy across distributed and non-uniform datasets. In addition to its superior segmentation performance, ConDistFL maintains computational and communication efficiency, ensuring its scalability for real-world applications. Furthermore, ConDistFL demonstrates remarkable generalizability, significantly outperforming existing FL methods in out-of-federation tests, even adapting to unseen contrast phases (e.g., non-contrast CT images) in our experiments. Extensive evaluations on 3D CT and 2D chest X-ray datasets show that ConDistFL is an efficient, adaptable solution for collaborative medical image segmentation in privacy-constrained settings.