Personalized Federated Learning via Feature Distribution Adaptation

TL;DR

This paper introduces pFedFDA, a personalized federated learning method that adapts global generative classifiers to local feature distributions, effectively handling distribution shifts and improving performance in data-scarce scenarios.

Contribution

The paper proposes a novel approach framing representation learning as a generative modeling task, enabling efficient personalization by adapting global classifiers to local data distributions.

Findings

Significant performance improvements over state-of-the-art methods.

Effective handling of complex distribution shifts.

Robustness in data-scarce settings.

Abstract

Federated learning (FL) is a distributed learning framework that leverages commonalities between distributed client datasets to train a global model. Under heterogeneous clients, however, FL can fail to produce stable training results. Personalized federated learning (PFL) seeks to address this by learning individual models tailored to each client. One approach is to decompose model training into shared representation learning and personalized classifier training. Nonetheless, previous works struggle to navigate the bias-variance trade-off in classifier learning, relying solely on limited local datasets or introducing costly techniques to improve generalization. In this work, we frame representation learning as a generative modeling task, where representations are trained with a classifier based on the global feature distribution. We then propose an algorithm, pFedFDA, that efficiently generates personalized models by adapting global generative classifiers to their local feature distributions. Through extensive computer vision benchmarks, we demonstrate that our method can adjust to complex distribution shifts with significant improvements over current state-of-the-art in data-scarce settings.