Variance Reduction for the Independent Metropolis Sampler

TL;DR

This paper introduces a variance reduction technique for the independent Metropolis sampler using control variates, and proposes an adaptive algorithm to improve proposal density, demonstrated on Bayesian regression problems.

Contribution

It presents a novel variance reduction method for the independent Metropolis sampler and an adaptive proposal strategy to minimize KL divergence with the target density.

Findings

Variance reduction achieves smaller asymptotic variance than i.i.d. sampling.

The adaptive algorithm effectively reduces KL divergence in practice.

Method applied successfully to Bayesian logistic and Gaussian process regression.

Abstract

Assume that we would like to estimate the expected value of a function $F$ with respect to an intractable density $\pi$, which is specified up to some unknown normalising constant. We prove that if $\pi$ is close enough under KL divergence to another density $q$, an independent Metropolis sampler estimator that obtains samples from $\pi$ with proposal density $q$, enriched with a variance reduction computational strategy based on control variates, achieves smaller asymptotic variance than i.i.d.\ sampling from $\pi$. The control variates construction requires no extra computational effort but assumes that the expected value of $F$ under $q$ is analytically available. We illustrate this result by calculating the marginal likelihood in a linear regression model with prior-likelihood conflict and a non-conjugate prior. Furthermore, we propose an adaptive independent Metropolis algorithm that adapts the proposal density such that its KL divergence with the target is being reduced. We demonstrate its applicability in a Bayesian logistic and Gaussian process regression problems and we rigorously justify our asymptotic arguments under easily verifiable and essentially minimal conditions.