FedAGHN: Personalized Federated Learning with Attentive Graph HyperNetworks

TL;DR

FedAGHN introduces an attentive graph hypernetwork approach to dynamically model client relationships in personalized federated learning, leading to improved personalized models and adaptable collaboration strategies.

Contribution

The paper proposes a novel Attentive Graph HyperNetwork framework for personalized federated learning, enabling dynamic modeling of client relationships for better personalization.

Findings

FedAGHN outperforms existing PFL methods in experiments.

The learned collaboration graphs effectively capture client relationships.

Visualizations demonstrate the adaptability of the collaboration graphs.

Abstract

Personalized Federated Learning (PFL) aims to address the statistical heterogeneity of data across clients by learning the personalized model for each client. Among various PFL approaches, the personalized aggregation-based approach conducts parameter aggregation in the server-side aggregation phase to generate personalized models, and focuses on learning appropriate collaborative relationships among clients for aggregation. However, the collaborative relationships vary in different scenarios and even at different stages of the FL process. To this end, we propose Personalized Federated Learning with Attentive Graph HyperNetworks (FedAGHN), which employs Attentive Graph HyperNetworks (AGHNs) to dynamically capture fine-grained collaborative relationships and generate client-specific personalized initial models. Specifically, AGHNs empower graphs to explicitly model the client-specific collaborative relationships, construct collaboration graphs, and introduce tunable attentive mechanism to derive the collaboration weights, so that the personalized initial models can be obtained by aggregating parameters over the collaboration graphs. Extensive experiments can demonstrate the superiority of FedAGHN. Moreover, a series of visualizations are presented to explore the effectiveness of collaboration graphs learned by FedAGHN.