Subsampled R\'enyi Differential Privacy and Analytical Moments Accountant

TL;DR

This paper derives a tight upper bound on RDP parameters for subsampled mechanisms, extending moments accounting to a broader class of differentially private algorithms, enhancing privacy analysis precision.

Contribution

It generalizes the moments accounting technique to subsampled RDP mechanisms, providing a more accurate privacy analysis tool for private machine learning algorithms.

Findings

Provides a tight upper bound on RDP for subsampled mechanisms.

Extends moments accounting technique beyond Gaussian mechanisms.

Improves privacy accounting accuracy in DP algorithms.

Abstract

We study the problem of subsampling in differential privacy (DP), a question that is the centerpiece behind many successful differentially private machine learning algorithms. Specifically, we provide a tight upper bound on the R\'enyi Differential Privacy (RDP) (Mironov, 2017) parameters for algorithms that: (1) subsample the dataset, and then (2) applies a randomized mechanism M to the subsample, in terms of the RDP parameters of M and the subsampling probability parameter. Our results generalize the moments accounting technique, developed by Abadi et al. (2016) for the Gaussian mechanism, to any subsampled RDP mechanism.