Hypernetworks for Model-Heterogeneous Personalized Federated Learning

TL;DR

This paper introduces MH-pFedHN, a hypernetwork-based framework for personalized federated learning that handles client model heterogeneity without external data, enhancing privacy, scalability, and generalization.

Contribution

The paper proposes a novel hypernetwork approach with multi-head sharing and optional global model integration for model-heterogeneous federated learning.

Findings

Achieves competitive accuracy across benchmarks.

Demonstrates strong generalization capabilities.

Provides a flexible, privacy-preserving framework.

Abstract

Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning strategies, which can limit their practicality and scalability. In this paper, we revisit hypernetwork-based methods and leverage their strong generalization capabilities to design a simple yet effective framework for heterogeneous personalized federated learning. Specifically, we propose MH-pFedHN, which leverages a server-side hypernetwork that takes client-specific embedding vectors as input and outputs personalized parameters tailored to each client's heterogeneous model. To promote knowledge sharing and reduce computation, we introduce a multi-head structure within the hypernetwork, allowing clients with similar model sizes to share heads. Furthermore, we further propose MH-pFedHNGD, which integrates an optional lightweight global model to improve generalization. Our framework does not rely on external datasets and does not require disclosure of client model architectures, thereby offering enhanced privacy and flexibility. Extensive experiments on multiple benchmarks and model settings demonstrate that our approach achieves competitive accuracy, strong generalization, and serves as a robust baseline for future research in model-heterogeneous personalized federated learning.