On the Use of Penalty MCMC for Differential Privacy

TL;DR

This paper explores the penalty MCMC algorithm's potential for differential privacy, demonstrating its favorable convergence and privacy properties in simple models and proposing schemes for distributed data and exponential family likelihoods.

Contribution

It analyzes the penalty algorithm's differential privacy capabilities and introduces privacy-preserving schemes for distributed data and exponential family models.

Findings

Algorithm has desirable convergence properties that scale with data size.

Proposed privacy schemes for distributed data owners.

Effective privacy preservation in exponential family models.

Abstract

We view the penalty algorithm of Ceperley and Dewing (1999), a Markov chain Monte Carlo (MCMC) algorithm for Bayesian inference, in the context of data privacy. Specifically, we study differential privacy of the penalty algorithm and advocate its use for data privacy. We show that in the simple model of independent observations the algorithm has desirable convergence and privacy properties that scale with data size. Two special cases are also investigated and privacy preserving schemes are proposed for those cases: (i) Data are distributed among several data owners who are interested in the inference of a common parameter while preserving their data privacy. (ii) The data likelihood belongs to an exponential family.