Bias correction of posterior means using MCMC outputs

TL;DR

This paper introduces algorithms to correct bias in posterior mean estimators derived from MCMC outputs, leveraging the Bayesian infinitesimal jackknife approximation for improved accuracy in various statistical models.

Contribution

It presents two novel algorithms for bias correction of posterior means using MCMC outputs, including a high-dimensional sparse data adaptation with iterative quasi-prior refinement.

Findings

Algorithms effectively reduce bias in posterior mean estimates.

Successful implementation in Weibull and logistic regression models.

Applicable to high-dimensional and sparse data settings.

Abstract

We propose algorithms for addressing the bias of the posterior mean when used as an estimator of parameters. These algorithms build upon the recently proposed Bayesian infinitesimal jackknife approximation (Giordano and Broderick (2023)) and can be implemented using the posterior covariance and third-order combined cumulants easily calculated from MCMC outputs. Two algorithms are introduced: The first algorithm utilises the output of a single-run MCMC with the original likelihood and prior to estimate the bias. A notable feature of the algorithm is that its ability to estimate definitional bias (Efron (2015)), which is crucial for Bayesian estimators. The second algorithm is designed for high-dimensional and sparse data settings, where ``quasi-prior'' for bias correction is introduced. The quasi-prior is iteratively refined using the output of the first algorithm as a measure of the residual bias at each step. These algorithms have been successfully implemented and tested for parameter estimation in the Weibull distribution and logistic regression in moderately high-dimensional settings.