Privacy Amplification Via Bernoulli Sampling

TL;DR

This paper analyzes how Bernoulli sampling can enhance privacy guarantees in differentially private algorithms, providing algorithms and bounds for the amplification effect in various applications.

Contribution

It introduces a method to compute privacy amplification factors for Bernoulli sampling and establishes bounds, advancing understanding of privacy-accuracy tradeoffs.

Findings

Provides an algorithm to compute amplification factors.

Establishes upper and lower bounds on privacy amplification.

Applicable to Bayesian inference and data compression.

Abstract

Balancing privacy and accuracy is a major challenge in designing differentially private machine learning algorithms. One way to improve this tradeoff for free is to leverage the noise in common data operations that already use randomness. Such operations include noisy SGD and data subsampling. The additional noise in these operations may amplify the privacy guarantee of the overall algorithm, a phenomenon known as privacy amplification. In this paper, we analyze the privacy amplification of sampling from a multidimensional Bernoulli distribution family given the parameter from a private algorithm. This setup has applications to Bayesian inference and to data compression. We provide an algorithm to compute the amplification factor, and we establish upper and lower bounds on this factor.