An Adaptive Differentially Private Federated Learning Framework with Bi-level Optimization

TL;DR

This paper introduces an adaptive federated learning framework that enhances model stability and accuracy under heterogeneous data and privacy constraints by employing dynamic gradient clipping, regularization, and noise suppression techniques.

Contribution

It proposes novel adaptive strategies for gradient clipping and update aggregation to improve federated learning performance with differential privacy in non-IID settings.

Findings

Improved convergence stability on CIFAR-10 and SVHN datasets.

Enhanced classification accuracy under privacy constraints.

Effective mitigation of gradient noise amplification.

Abstract

Federated learning enables collaborative model training across distributed clients while preserving data privacy. However, in practical deployments, device heterogeneity, non-independent, and identically distributed (Non-IID) data often lead to highly unstable and biased gradient updates. When differential privacy is enforced, conventional fixed gradient clipping and Gaussian noise injection may further amplify gradient perturbations, resulting in training oscillation and performance degradation and degraded model performance. To address these challenges, we propose an adaptive differentially private federated learning framework that explicitly targets model efficiency under heterogeneous and privacy-constrained settings. On the client side, a lightweight local compressed module is introduced to regularize intermediate representations and constrain gradient variability, thereby mitigating noise amplification during local optimization. On the server side, an adaptive gradient clipping strategy dynamically adjusts clipping thresholds based on historical update statistics to avoid over-clipping and noise domination. Furthermore, a constraint-aware aggregation mechanism is designed to suppress unreliable or noise-dominated client updates and stabilize global optimization. Extensive experiments on CIFAR-10 and SVHN demonstrate improved convergence stability and classification accuracy.