Cluster-driven Graph Federated Learning over Multiple Domains

TL;DR

This paper introduces FedCG, a novel federated learning framework that uses clustering and graph convolutional networks to effectively handle data heterogeneity across multiple domains, achieving state-of-the-art results.

Contribution

FedCG combines domain clustering with GCNs to improve knowledge sharing and adapt to unseen domains in federated learning, addressing heterogeneity more effectively than existing methods.

Findings

Achieves state-of-the-art performance on multiple FL benchmarks.

Effectively clusters clients into domains with unsupervised learning.

Enables adaptation to unseen test domains using soft-assignment scores.

Abstract

Federated Learning (FL) deals with learning a central model (i.e. the server) in privacy-constrained scenarios, where data are stored on multiple devices (i.e. the clients). The central model has no direct access to the data, but only to the updates of the parameters computed locally by each client. This raises a problem, known as statistical heterogeneity, because the clients may have different data distributions (i.e. domains). This is only partly alleviated by clustering the clients. Clustering may reduce heterogeneity by identifying the domains, but it deprives each cluster model of the data and supervision of others. Here we propose a novel Cluster-driven Graph Federated Learning (FedCG). In FedCG, clustering serves to address statistical heterogeneity, while Graph Convolutional Networks (GCNs) enable sharing knowledge across them. FedCG: i) identifies the domains via an FL-compliant clustering and instantiates domain-specific modules (residual branches) for each domain; ii) connects the domain-specific modules through a GCN at training to learn the interactions among domains and share knowledge; and iii) learns to cluster unsupervised via teacher-student classifier-training iterations and to address novel unseen test domains via their domain soft-assignment scores. Thanks to the unique interplay of GCN over clusters, FedCG achieves the state-of-the-art on multiple FL benchmarks.