Tackling Privacy Heterogeneity in Differentially Private Federated Learning

TL;DR

This paper addresses the challenge of privacy heterogeneity in federated learning by developing a privacy-aware client selection method that improves model accuracy under diverse privacy constraints.

Contribution

It introduces the first systematic study and theoretical analysis of privacy-aware client selection in DP-FL, proposing an adaptive optimization strategy.

Findings

Achieves up to 10% test accuracy improvement on CIFAR-10.

Provides a theoretical convergence analysis considering privacy heterogeneity.

Demonstrates effectiveness of the proposed method through extensive experiments.

Abstract

Differentially private federated learning (DP-FL) enables clients to collaboratively train machine learning models while preserving the privacy of their local data. However, most existing DP-FL approaches assume that all clients share a uniform privacy budget, an assumption that does not hold in real-world scenarios where privacy requirements vary widely. This privacy heterogeneity poses a significant challenge: conventional client selection strategies, which typically rely on data quantity, cannot distinguish between clients providing high-quality updates and those introducing substantial noise due to strict privacy constraints. To address this gap, we present the first systematic study of privacy-aware client selection in DP-FL. We establish a theoretical foundation by deriving a convergence analysis that quantifies the impact of privacy heterogeneity on training error. Building on this analysis, we propose a privacy-aware client selection strategy, formulated as a convex optimization problem, that adaptively adjusts selection probabilities to minimize training error. Extensive experiments on benchmark datasets demonstrate that our approach achieves up to a 10% improvement in test accuracy on CIFAR-10 compared to existing baselines under heterogeneous privacy budgets. These results highlight the importance of incorporating privacy heterogeneity into client selection for practical and effective federated learning.