A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning

TL;DR

FedHydra is a novel one-shot federated learning framework that effectively handles both model and data heterogeneity with a two-stage learning mechanism, outperforming existing methods in diverse settings.

Contribution

Introduces FedHydra, a unified OSFL framework that addresses both model and data heterogeneity through model stratification and heterogeneity-aware aggregation.

Findings

Outperforms five SOTA baselines on four benchmark datasets.

Effectively mitigates model and data heterogeneity issues.

Works well in both homogeneous and heterogeneous settings.

Abstract

One-Shot Federated Learning (OSFL) restricts communication between the server and clients to a single round, significantly reducing communication costs and minimizing privacy leakage risks compared to traditional Federated Learning (FL), which requires multiple rounds of communication. However, existing OSFL frameworks remain vulnerable to distributional heterogeneity, as they primarily focus on model heterogeneity while neglecting data heterogeneity. To bridge this gap, we propose FedHydra, a unified, data-free, OSFL framework designed to effectively address both model and data heterogeneity. Unlike existing OSFL approaches, FedHydra introduces a novel two-stage learning mechanism. Specifically, it incorporates model stratification and heterogeneity-aware stratified aggregation to mitigate the challenges posed by both model and data heterogeneity. By this design, the data and model heterogeneity issues are simultaneously monitored from different aspects during learning. Consequently, FedHydra can effectively mitigate both issues by minimizing their inherent conflicts. We compared FedHydra with five SOTA baselines on four benchmark datasets. Experimental results show that our method outperforms the previous OSFL methods in both homogeneous and heterogeneous settings. The code is available at https://github.com/Jun-B0518/FedHydra.