Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise

TL;DR

This paper introduces Adap DP-FL, a novel differentially private federated learning method that adaptively adjusts noise and gradient clipping based on client heterogeneity and convergence, improving privacy-utility trade-offs.

Contribution

It proposes an adaptive noise addition and gradient clipping scheme tailored to federated learning's heterogeneity and convergence issues, enhancing privacy without sacrificing performance.

Findings

Outperforms previous methods significantly in experiments

Effectively balances privacy and model accuracy

Adapts noise levels based on training dynamics

Abstract

Federated learning seeks to address the issue of isolated data islands by making clients disclose only their local training models. However, it was demonstrated that private information could still be inferred by analyzing local model parameters, such as deep neural network model weights. Recently, differential privacy has been applied to federated learning to protect data privacy, but the noise added may degrade the learning performance much. Typically, in previous work, training parameters were clipped equally and noises were added uniformly. The heterogeneity and convergence of training parameters were simply not considered. In this paper, we propose a differentially private scheme for federated learning with adaptive noise (Adap DP-FL). Specifically, due to the gradient heterogeneity, we conduct adaptive gradient clipping for different clients and different rounds; due to the gradient convergence, we add decreasing noises accordingly. Extensive experiments on real-world datasets demonstrate that our Adap DP-FL outperforms previous methods significantly.