Exact Sampling of Gibbs Measures with Estimated Losses

TL;DR

This paper introduces a modified PDMP sampler for exact sampling of Gibbs measures with estimated losses, overcoming dependence on pseudo-observations and improving inference in complex models.

Contribution

It proposes a novel PDMP-based sampling method that eliminates the dependence on the number of pseudo-observations in Gibbs measure inference.

Findings

Standard MCMC methods exhibit slow concentration with pseudo-observations.

Modified PDMP samplers remove dependence on pseudo-observation count.

Empirical results confirm improved sampling efficiency in complex models.

Abstract

In recent years, the shortcomings of Bayesian posteriors as inferential devices have received increased attention. A popular strategy for fixing them has been to instead target a Gibbs measure based on losses that connect a parameter of interest to observed data. However, existing theory for such inference procedures assumes these losses are analytically available, while in many situations these losses must be stochastically estimated using pseudo-observations. In such cases, we show that when standard Markov Chain Monte Carlo algorithms are used to produce posterior samples, the resulting posterior exhibits strong dependence on the number of pseudo-observations: unless the number of pseudo-observations diverge sufficiently fast the resulting posterior will concentrate very slowly. However, we show that in many situations it is feasible to alleviate this dependence entirely using a modified piecewise deterministic Markov process (PDMP) sampler, and we formally and empirically show that these samplers produce posterior draws that have no dependence on the number of pseudo-observations used to estimate the loss within the Gibbs Measure. We apply our results to three examples that feature intractable likelihoods and model misspecification.