SOMA: A Novel Sampler for Bayesian Inference from Privatized Data

TL;DR

This paper introduces SOMA, a new sampler for Bayesian inference from privatized data that enhances efficiency by sharing proposals across components, leading to faster convergence and better acceptance rates.

Contribution

The paper presents SOMA, a novel sampler that improves upon standard DAMCMC by sharing proposals across components, with proven theoretical guarantees and demonstrated empirical efficiency.

Findings

SOMA achieves higher acceptance rates than traditional methods.

Theoretical bounds on SOMA's acceptance probability are established.

Experiments show SOMA's efficiency on synthetic and real census data.

Abstract

Making valid statistical inferences from privatized data is a key challenge in modern analysis. In Bayesian settings, data augmentation MCMC (DAMCMC) methods impute unobserved confidential data given noisy privatized summaries, enabling principled uncertainty quantification. However, standard DAMCMC often suffers from slow mixing due to component-wise Metropolis-within-Gibbs updates. We propose the Single-Offer-Multiple-Attempts (SOMA) sampler. This novel algorithm improves acceptance rates by generating a single proposal and simultaneously evaluating its suitability to replace all components. By sharing proposals across components, SOMA rejects fewer proposal points. We prove lower bounds on SOMA's acceptance probability and establish convergence rates in the two-component case. Experiments on synthetic and real census data with linear regression and other models confirm SOMA's efficiency gains.