Convergence-Privacy-Fairness Trade-Off in Personalized Federated Learning

TL;DR

This paper introduces DP-Ditto, a differentially private personalized federated learning method that balances privacy, convergence, and fairness, outperforming existing models in fairness and accuracy.

Contribution

It extends Ditto with differential privacy, analyzes the privacy-convergence-fairness trade-off, and provides optimal aggregation strategies under privacy constraints.

Findings

DP-Ditto outperforms state-of-the-art PFL models in fairness by over 32.71%.

DP-Ditto achieves up to 9.66% higher accuracy compared to other DP-perturbed models.

The analysis guides optimal global aggregation under privacy budgets.

Abstract

Personalized federated learning (PFL), e.g., the renowned Ditto, strikes a balance between personalization and generalization by conducting federated learning (FL) to guide personalized learning (PL). While FL is unaffected by personalized model training, in Ditto, PL depends on the outcome of the FL. However, the clients' concern about their privacy and consequent perturbation of their local models can affect the convergence and (performance) fairness of PL. This paper presents PFL, called DP-Ditto, which is a non-trivial extension of Ditto under the protection of differential privacy (DP), and analyzes the trade-off among its privacy guarantee, model convergence, and performance distribution fairness. We also analyze the convergence upper bound of the personalized models under DP-Ditto and derive the optimal number of global aggregations given a privacy budget. Further, we analyze the performance fairness of the personalized models, and reveal the feasibility of optimizing DP-Ditto jointly for convergence and fairness. Experiments validate our analysis and demonstrate that DP-Ditto can surpass the DP-perturbed versions of the state-of-the-art PFL models, such as FedAMP, pFedMe, APPLE, and FedALA, by over 32.71% in fairness and 9.66% in accuracy.