Insufficient Gibbs Sampling

TL;DR

This paper introduces a Gibbs sampling approach to estimate posterior distributions of parameters using only robust, aggregated statistics, addressing privacy constraints in data analysis.

Contribution

It proposes a novel method to perform Bayesian inference with limited data summaries, enabling posterior sampling and model comparison under privacy-preserving conditions.

Findings

Effective in toy examples and real income data

Allows estimation of Bayes factors from summary statistics

Highlights limitations and potential of the approach

Abstract

In some applied scenarios, the availability of complete data is restricted, often due to privacy concerns; only aggregated, robust and inefficient statistics derived from the data are made accessible. These robust statistics are not sufficient, but they demonstrate reduced sensitivity to outliers and offer enhanced data protection due to their higher breakdown point. We consider a parametric framework and propose a method to sample from the posterior distribution of parameters conditioned on various robust and inefficient statistics: specifically, the pairs (median, MAD) or (median, IQR), or a collection of quantiles. Our approach leverages a Gibbs sampler and simulates latent augmented data, which facilitates simulation from the posterior distribution of parameters belonging to specific families of distributions. A by-product of these samples from the joint posterior distribution of parameters and data given the observed statistics is that we can estimate Bayes factors based on observed statistics via bridge sampling. We validate and outline the limitations of the proposed methods through toy examples and an application to real-world income data.