Deploying Models to Non-participating Clients in Federated Learning without Fine-tuning: A Hypernetwork-based Approach

TL;DR

HyperFedZero introduces a hypernetwork-based method to generate personalized models for non-participating federated clients, effectively handling data heterogeneity without fine-tuning, and demonstrating superior performance with low overhead.

Contribution

The paper proposes HyperFedZero, a novel hypernetwork approach that dynamically generates client-specific models based on distribution-aware embeddings, addressing non-participating client adaptation in federated learning.

Findings

Outperforms existing methods across multiple datasets and models.

Maintains low computational, storage, and communication overhead.

Ablation studies confirm the importance of each component.

Abstract

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving collaborative learning, yet data heterogeneity remains a critical challenge. While existing methods achieve progress in addressing data heterogeneity for participating clients, they fail to generalize to non-participating clients with in-domain distribution shifts and resource constraints. To mitigate this issue, we present HyperFedZero, a novel method that dynamically generates specialized models via a hypernetwork conditioned on distribution-aware embeddings. Our approach explicitly incorporates distribution-aware inductive biases into the model's forward pass, extracting robust distribution embeddings using a NoisyEmbed-enhanced extractor with a Balancing Penalty, effectively preventing feature collapse. The hypernetwork then leverages these embeddings to generate specialized models chunk-by-chunk for non-participating clients, ensuring adaptability to their unique data distributions. Extensive experiments on multiple datasets and models demonstrate HyperFedZero's remarkable performance, surpassing competing methods consistently with minimal computational, storage, and communication overhead. Moreover, ablation studies and visualizations further validate the necessity of each component, confirming meaningful adaptations and validating the effectiveness of HyperFedZero.