Infinitely Divisible Noise in the Low Privacy Regime

TL;DR

This paper introduces a new infinitely divisible noise distribution for differential privacy in federated learning, achieving lower error in the low privacy regime by exponentially reducing expected error as privacy constraints loosen.

Contribution

It proposes the first infinitely divisible noise distribution that attains -privacy with exponentially decreasing expected error, improving privacy-utility trade-offs.

Findings

New noise distribution achieves -privacy with lower error.

Expected error decreases exponentially with  in the low privacy regime.

Enhances federated learning privacy mechanisms.

Abstract

Federated learning, in which training data is distributed among users and never shared, has emerged as a popular approach to privacy-preserving machine learning. Cryptographic techniques such as secure aggregation are used to aggregate contributions, like a model update, from all users. A robust technique for making such aggregates differentially private is to exploit infinite divisibility of the Laplace distribution, namely, that a Laplace distribution can be expressed as a sum of i.i.d. noise shares from a Gamma distribution, one share added by each user. However, Laplace noise is known to have suboptimal error in the low privacy regime for $\varepsilon$-differential privacy, where $\varepsilon > 1$ is a large constant. In this paper we present the first infinitely divisible noise distribution for real-valued data that achieves $\varepsilon$-differential privacy and has expected error that decreases exponentially with $\varepsilon$.