Differential privacy guarantees of Markov chain Monte Carlo algorithms

TL;DR

This paper develops differential privacy guarantees for MCMC algorithms, including Langevin-based methods, by analyzing their convergence and output properties to ensure privacy of the target distribution and entire chain trajectories.

Contribution

It introduces a novel methodology using Girsanov's theorem for privacy analysis of unbounded, non-convex MCMC methods, providing concrete privacy guarantees.

Findings

Established DP guarantees on MCMC outputs and estimators.

Derived bounds on privacy for entire chain trajectories.

Provided guidelines for privacy-preserving MCMC implementations.

Abstract

This paper aims to provide differential privacy (DP) guarantees for Markov chain Monte Carlo (MCMC) algorithms. In a first part, we establish DP guarantees on samples output by MCMC algorithms as well as Monte Carlo estimators associated with these methods under assumptions on the convergence properties of the underlying Markov chain. In particular, our results highlight the critical condition of ensuring the target distribution is differentially private itself. In a second part, we specialise our analysis to the unadjusted Langevin algorithm and stochastic gradient Langevin dynamics and establish guarantees on their (R\'enyi) DP. To this end, we develop a novel methodology based on Girsanov's theorem combined with a perturbation trick to obtain bounds for an unbounded domain and in a non-convex setting. We establish: (i) uniform in $n$ privacy guarantees when the state of the chain after $n$ iterations is released, (ii) bounds on the privacy of the entire chain trajectory. These findings provide concrete guidelines for privacy-preserving MCMC.