Heterogeneous Federated Learning via Personalized Generative Networks

TL;DR

This paper introduces a method for federated learning that uses personalized generative networks to reduce client heterogeneity, improving convergence and model generalization in the presence of data shifts.

Contribution

It proposes training client-specific generators at the server to generate personalized data, addressing heterogeneity and concept shifts in federated learning.

Findings

Theoretical proof linking heterogeneity minimization to improved convergence.

Experimental results show enhanced global model performance on synthetic and real data.

Method effectively reduces conflicts between local models, leading to better generalization.

Abstract

Federated Learning (FL) allows several clients to construct a common global machine-learning model without having to share their data. FL, however, faces the challenge of statistical heterogeneity between the client's data, which degrades performance and slows down the convergence toward the global model. In this paper, we provide theoretical proof that minimizing heterogeneity between clients facilitates the convergence of a global model for every single client. This becomes particularly important under empirical concept shifts among clients, rather than merely considering imbalanced classes, which have been studied until now. Therefore, we propose a method for knowledge transfer between clients where the server trains client-specific generators. Each generator generates samples for the corresponding client to remove the conflict with other clients' models. Experiments conducted on synthetic and real data, along with a theoretical study, support the effectiveness of our method in constructing a well-generalizable global model by reducing the conflict between local models.