FedCP: Separating Feature Information for Personalized Federated Learning via Conditional Policy

TL;DR

FedCP introduces a sample-specific approach in personalized federated learning by separating global and personalized feature information through a conditional policy, leading to improved performance and robustness.

Contribution

The paper proposes FedCP, a novel method that generates sample-specific conditional policies to distinguish and process global and personalized feature information in pFL.

Findings

FedCP outperforms 11 state-of-the-art methods by up to 6.69%.

FedCP maintains performance when clients drop out.

Effective in both computer vision and NLP tasks.

Abstract

Recently, personalized federated learning (pFL) has attracted increasing attention in privacy protection, collaborative learning, and tackling statistical heterogeneity among clients, e.g., hospitals, mobile smartphones, etc. Most existing pFL methods focus on exploiting the global information and personalized information in the client-level model parameters while neglecting that data is the source of these two kinds of information. To address this, we propose the Federated Conditional Policy (FedCP) method, which generates a conditional policy for each sample to separate the global information and personalized information in its features and then processes them by a global head and a personalized head, respectively. FedCP is more fine-grained to consider personalization in a sample-specific manner than existing pFL methods. Extensive experiments in computer vision and natural language processing domains show that FedCP outperforms eleven state-of-the-art methods by up to 6.69%. Furthermore, FedCP maintains its superiority when some clients accidentally drop out, which frequently happens in mobile settings. Our code is public at https://github.com/TsingZ0/FedCP.