Personalized Federated Learning with Feature Alignment and Classifier Collaboration

TL;DR

This paper proposes a personalized federated learning method that enhances local feature representations with global knowledge and optimizes classifier collaboration, improving model generalization across heterogeneous data.

Contribution

It introduces explicit local-global feature alignment and a novel classifier weight optimization approach for personalized federated learning.

Findings

Improved accuracy on benchmark datasets.

Effective handling of data heterogeneity.

Enhanced model generalization.

Abstract

Data heterogeneity is one of the most challenging issues in federated learning, which motivates a variety of approaches to learn personalized models for participating clients. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. However, previous works do not utilize the global knowledge during local representation learning and also neglect the fine-grained collaboration between local classifier heads, which limit the model generalization ability. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation. Moreover, we quantify the benefit of classifier combination for each client as a function of the combining weights and derive an optimization problem for estimating optimal weights. Finally, extensive evaluation results on benchmark datasets with various heterogeneous data scenarios demonstrate the effectiveness of our proposed method. Code is available at https://github.com/JianXu95/FedPAC