Local Layer-wise Differential Privacy in Federated Learning

TL;DR

LaDP introduces a layer-wise adaptive noise injection method in federated learning that enhances privacy while maintaining high model utility, outperforming existing solutions in accuracy and privacy protection.

Contribution

This paper presents LaDP, a novel layer-wise differential privacy mechanism for federated learning that optimizes privacy-utility tradeoff through dynamic, importance-based noise injection.

Findings

Reduces noise injection by 46.14% compared to SOTA methods.

Improves model accuracy by 102.99% over baseline.

Enhances robustness against reconstruction attacks, increasing FID by over 12.84%.

Abstract

Federated Learning (FL) enables collaborative model training without direct data sharing, yet it remains vulnerable to privacy attacks such as model inversion and membership inference. Existing differential privacy (DP) solutions for FL often inject noise uniformly across the entire model, degrading utility while providing suboptimal privacy-utility tradeoffs. To address this, we propose LaDP, a novel layer-wise adaptive noise injection mechanism for FL that optimizes privacy protection while preserving model accuracy. LaDP leverages two key insights: (1) neural network layers contribute unevenly to model utility, and (2) layer-wise privacy leakage can be quantified via KL divergence between local and global model distributions. LaDP dynamically injects noise into selected layers based on their privacy sensitivity and importance to model performance. We provide a rigorous theoretical analysis, proving that LaDP satisfies $(\epsilon, \delta)$-DP guarantees and converges under bounded noise. Extensive experiments on CIFAR-10/100 datasets demonstrate that LaDP reduces noise injection by 46.14% on average compared to state-of-the-art (SOTA) methods while improving accuracy by 102.99%. Under the same privacy budget, LaDP outperforms SOTA solutions like Dynamic Privacy Allocation LDP and AdapLDP by 25.18% and 6.1% in accuracy, respectively. Additionally, LaDP robustly defends against reconstruction attacks, increasing the FID of the reconstructed private data by $>$12.84% compared to all baselines. Our work advances the practical deployment of privacy-preserving FL with minimal utility loss.