Calibration procedures for approximate Bayesian credible sets

TL;DR

This paper introduces two calibration methods to evaluate the coverage accuracy of approximate Bayesian credible sets, especially those derived from Monte Carlo methods, using simulated data and importance sampling.

Contribution

The authors propose novel calibration procedures for assessing the coverage of approximate Bayesian credible sets, applicable when the posterior is approximated and not proportional to the ideal likelihood and prior.

Findings

The calibration methods effectively estimate the true coverage of approximate credible sets.

Application to four examples demonstrates the methods' practical utility.

The semi-parametric logistic regression approach provides reliable coverage estimates.

Abstract

We develop and apply two calibration procedures for checking the coverage of approximate Bayesian credible sets including intervals estimated using Monte Carlo methods. The user has an ideal prior and likelihood, but generates a credible set for an approximate posterior which is not proportional to the product of ideal likelihood and prior. We estimate the realised posterior coverage achieved by the approximate credible set. This is the coverage of the unknown ``true'' parameter if the data are a realisation of the user's ideal observation model conditioned on the parameter, and the parameter is a draw from the user's ideal prior. In one approach we estimate the posterior coverage at the data by making a semi-parametric logistic regression of binary coverage outcomes on simulated data against summary statistics evaluated on simulated data. In another we use Importance Sampling from the approximate posterior, windowing simulated data to fall close to the observed data. We illustrate our methods on four examples.