A Game-Theoretic Framework for Privacy-Aware Client Sampling in Federated Learning

TL;DR

This paper introduces FedPCS, a game-theoretic framework for privacy-aware client sampling in federated learning, improving model accuracy and efficiency by balancing privacy, incentives, and sampling strategies.

Contribution

It develops a novel privacy-aware client sampling method using a Stackelberg game model, mean-field estimation, and theoretical convergence guarantees, advancing federated learning strategies.

Findings

FedPCS reduces the Price of Anarchy compared to random sampling.

The framework achieves superior accuracy on IID and Non-IID datasets.

Theoretical analysis guarantees robustness and convergence of the proposed method.

Abstract

This paper aims to design a Privacy-aware Client Sampling framework in Federated learning, named FedPCS, to tackle the heterogeneous client sampling issues and improve model performance. First, we obtain a pioneering upper bound for the accuracy loss of the FL model with privacy-aware client sampling probabilities. Based on this, we model the interactions between the central server and participating clients as a two-stage Stackelberg game. In Stage I, the central server designs the optimal time-dependent reward for cost minimization by considering the trade-off between the accuracy loss of the FL model and the rewards allocated. In Stage II, each client determines the correction factor that dynamically adjusts its privacy budget based on the reward allocated to maximize its utility. To surmount the obstacle of approximating other clients' private information, we introduce the mean-field estimator to estimate the average privacy budget. We analytically demonstrate the existence and convergence of the fixed point for the mean-field estimator and derive the Stackelberg Nash Equilibrium to obtain the optimal strategy profile. By rigorously theoretical convergence analysis, we guarantee the robustness of FedPCS. Moreover, considering the conventional sampling strategy in privacy-preserving FL, we prove that the random sampling approach's PoA can be arbitrarily large. To remedy such efficiency loss, we show that the proposed privacy-aware client sampling strategy successfully reduces PoA, which is upper bounded by a reachable constant. To address the challenge of varying privacy requirements throughout different training phases in FL, we extend our model and analysis and derive the adaptive optimal sampling ratio for the central server. Experimental results on different datasets demonstrate the superiority of FedPCS compared with the existing SOTA FL strategies under IID and Non-IID datasets.