Avoiding Pitfalls for Privacy Accounting of Subsampled Mechanisms under Composition

TL;DR

This paper clarifies misconceptions in privacy accounting for subsampled differentially private mechanisms, emphasizing the importance of accurate composition analysis and differences between sampling schemes for tighter privacy guarantees.

Contribution

It corrects common assumptions about privacy guarantees under composition and compares Poisson subsampling with sampling without replacement, providing more accurate privacy bounds.

Findings

Self-composition of worst-case datasets can be inaccurate for subsampled mechanisms.

Privacy guarantees differ significantly between Poisson subsampling and sampling without replacement.

Incorrect assumptions can lead to underestimating privacy loss in practical settings.

Abstract

We consider the problem of computing tight privacy guarantees for the composition of subsampled differentially private mechanisms. Recent algorithms can numerically compute the privacy parameters to arbitrary precision but must be carefully applied. Our main contribution is to address two common points of confusion. First, some privacy accountants assume that the privacy guarantees for the composition of a subsampled mechanism are determined by self-composing the worst-case datasets for the uncomposed mechanism. We show that this is not true in general. Second, Poisson subsampling is sometimes assumed to have similar privacy guarantees compared to sampling without replacement. We show that the privacy guarantees may in fact differ significantly between the two sampling schemes. In particular, we give an example of hyperparameters that result in $\varepsilon \approx 1$ for Poisson subsampling and $\varepsilon > 10$ for sampling without replacement. This occurs for some parameters that could realistically be chosen for DP-SGD.