FedBiCross: A Bi-Level Optimization Framework to Tackle Non-IID Challenges in Data-Free One-Shot Federated Learning on Medical Data

TL;DR

FedBiCross is a novel personalized federated learning framework that addresses non-IID challenges in data-free one-shot medical data scenarios by clustering clients and selectively leveraging cross-cluster knowledge.

Contribution

It introduces a three-stage bi-level optimization approach for personalized federated learning under non-IID data without sharing raw data.

Findings

FedBiCross outperforms existing methods on four medical image datasets.

The framework effectively handles different degrees of non-IID data.

Personalized distillation improves client-specific model performance.

Abstract

Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. However, existing methods aggregate predictions from all clients to form a global teacher. Under non-IID data, conflicting predictions cancel out during averaging, yielding near-uniform soft labels that provide weak supervision for distillation. We propose FedBiCross, a personalized OSFL framework with three stages: (1) clustering clients by model output similarity to form coherent sub-ensembles, (2) bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and (3) personalized distillation for client-specific adaptation. Experiments on four medical image datasets demonstrate that FedBiCross consistently outperforms state-of-the-art baselines across different non-IID degrees.