Markov Chain Importance Sampling -- a highly efficient estimator for MCMC

TL;DR

This paper introduces Markov chain importance sampling (MCIS), a new estimator that efficiently utilizes rejected proposals in Markov chain algorithms, improving accuracy and enabling normalizing constant estimation in Bayesian methods.

Contribution

The paper presents MCIS, a novel estimator for Markov chain algorithms that enhances efficiency by leveraging rejected proposals and corrects discretization errors in Langevin algorithms.

Findings

Satisfies a central limit theorem

Reduces error per CPU cycle significantly

Enables estimation of normalizing constants

Abstract

Markov chain (MC) algorithms are ubiquitous in machine learning and statistics and many other disciplines. Typically, these algorithms can be formulated as acceptance rejection methods. In this work we present a novel estimator applicable to these methods, dubbed Markov chain importance sampling (MCIS), which efficiently makes use of rejected proposals. For the unadjusted Langevin algorithm, it provides a novel way of correcting the discretization error. Our estimator satisfies a central limit theorem and improves on error per CPU cycle, often to a large extent. As a by-product it enables estimating the normalizing constant, an important quantity in Bayesian machine learning and statistics.