Zero Variance Markov Chain Monte Carlo for Bayesian Estimators

TL;DR

This paper introduces a zero-variance variance reduction technique for MCMC estimators, providing theoretical guarantees and demonstrating its effectiveness on Bayesian models like probit, logit, and GARCH.

Contribution

It proposes a novel zero-variance based variance reduction method for MCMC estimators with proven asymptotic unbiasedness and a central limit theorem.

Findings

The zero-variance estimator is asymptotically unbiased.

A central limit theorem is established for the estimator.

The method is successfully applied to Bayesian probit, logit, and GARCH models.

Abstract

Interest is in evaluating, by Markov chain Monte Carlo (MCMC) simulation, the expected value of a function with respect to a, possibly unnormalized, probability distribution. A general purpose variance reduction technique for the MCMC estimator, based on the zero-variance principle introduced in the physics literature, is proposed. Conditions for asymptotic unbiasedness of the zero-variance estimator are derived. A central limit theorem is also proved under regularity conditions. The potential of the idea is illustrated with real applications to probit, logit and GARCH Bayesian models. For all these models, a central limit theorem and unbiasedness for the zero-variance estimator are proved (see the supplementary material available on-line).