Differentially Private Federated Learning without Noise Addition: When is it Possible?

TL;DR

This paper investigates the theoretical possibility of achieving differential privacy in federated learning with secure aggregation without adding extra noise, identifying conditions where it is feasible and discussing practical limitations.

Contribution

It formally characterizes when secure aggregation can provide differential privacy without noise addition and highlights the practical challenges in meeting these conditions.

Findings

Gaussian randomness in aggregation can provide DP with eigenvalue bounds

Practical conditions for noise-free DP are unlikely to hold in real scenarios

Leveraging inherent randomness may reduce additional noise needed for DP

Abstract

Federated Learning (FL) with Secure Aggregation (SA) has gained significant attention as a privacy preserving framework for training machine learning models while preventing the server from learning information about users' data from their individual encrypted model updates. Recent research has extended privacy guarantees of FL with SA by bounding the information leakage through the aggregate model over multiple training rounds thanks to leveraging the "noise" from other users' updates. However, the privacy metric used in that work (mutual information) measures the on-average privacy leakage, without providing any privacy guarantees for worse-case scenarios. To address this, in this work we study the conditions under which FL with SA can provide worst-case differential privacy guarantees. Specifically, we formally identify the necessary condition that SA can provide DP without addition noise. We then prove that when the randomness inside the aggregated model update is Gaussian with non-singular covariance matrix, SA can provide differential privacy guarantees with the level of privacy $\epsilon$ bounded by the reciprocal of the minimum eigenvalue of the covariance matrix. However, we further demonstrate that in practice, these conditions are almost unlikely to hold and hence additional noise added in model updates is still required in order for SA in FL to achieve DP. Lastly, we discuss the potential solution of leveraging inherent randomness inside aggregated model update to reduce the amount of addition noise required for DP guarantee.