R\'enyi Differential Privacy of the Sampled Gaussian Mechanism

TL;DR

This paper provides a precise, numerically stable method to compute the Renyi Differential Privacy of the Sampled Gaussian Mechanism, enhancing understanding of its privacy amplification effects in machine learning.

Contribution

It unifies existing results on SGM's privacy properties and introduces a nearly tight, closed-form bound for its Renyi Differential Privacy.

Findings

Developed a numerically stable procedure for Renyi DP computation

Proved a nearly tight closed-form privacy bound

Filled gaps in previous SGM privacy analyses

Abstract

The Sampled Gaussian Mechanism (SGM)---a composition of subsampling and the additive Gaussian noise---has been successfully used in a number of machine learning applications. The mechanism's unexpected power is derived from privacy amplification by sampling where the privacy cost of a single evaluation diminishes quadratically, rather than linearly, with the sampling rate. Characterizing the precise privacy properties of SGM motivated development of several relaxations of the notion of differential privacy. This work unifies and fills in gaps in published results on SGM. We describe a numerically stable procedure for precise computation of SGM's R\'enyi Differential Privacy and prove a nearly tight (within a small constant factor) closed-form bound.