Privacy-Aware Rejection Sampling

TL;DR

This paper analyzes how rejection sampling's runtime can leak private information in differential privacy, quantifies the privacy loss, and proposes modifications to ensure data-independent runtime for enhanced privacy guarantees.

Contribution

It characterizes the privacy cost of rejection sampling runtimes and introduces three modifications to prevent timing attacks, including an approximate and two perfect samplers.

Findings

Rejection sampling runtime leaks private information unless acceptance probability is constant.

The privacy loss can be quantified in terms of $(\epsilon,\delta)$-DP and $f$-DP.

Proposed modifications can make rejection sampling's runtime independent of data, improving privacy protection.

Abstract

Differential privacy (DP) offers strong theoretical privacy guarantees, but implementations of DP mechanisms may be vulnerable to side-channel attacks, such as timing attacks. When sampling methods such as MCMC or rejection sampling are used to implement a mechanism, the runtime can leak private information. We characterize the additional privacy cost due to the runtime of a rejection sampler in terms of both $(\epsilon,\delta)$-DP as well as $f$-DP. We also show that unless the acceptance probability is constant across databases, the runtime of a rejection sampler does not satisfy $\epsilon$-DP for any $\epsilon$. We show that there is a similar breakdown in privacy with adaptive rejection samplers. We propose three modifications to the rejection sampling algorithm, with varying assumptions, to protect against timing attacks by making the runtime independent of the data. The modification with the weakest assumptions is an approximate sampler, introducing a small increase in the privacy cost, whereas the other modifications give perfect samplers. We also use our techniques to develop an adaptive rejection sampler for log-H\"{o}lder densities, which also has data-independent runtime. We give several examples of DP mechanisms that fit the assumptions of our methods and can thus be implemented using our samplers.