PGFed: Personalize Each Client's Global Objective for Federated Learning

TL;DR

PGFed introduces a personalized federated learning framework that explicitly personalizes each client's global objective by adaptively aggregating empirical risks, improving performance over existing methods while maintaining efficiency.

Contribution

This work proposes PGFed, a novel personalized FL framework that explicitly personalizes client objectives through adaptive risk aggregation, reducing communication overhead and enhancing model performance.

Findings

PGFed outperforms previous state-of-the-art methods on four datasets.

PGFed effectively personalizes models with lower communication costs.

PGFedMo further improves efficiency using a momentum-based upgrade.

Abstract

Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at https://github.com/ljaiverson/pgfed.