Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget

TL;DR

This paper proposes an approximate Metropolis-Hastings method that reduces computational costs for large datasets by using sequential hypothesis testing, balancing bias and variance for more efficient Bayesian sampling.

Contribution

It introduces a novel approximate MH rule based on sequential hypothesis testing, enabling faster sampling with controlled bias and increased efficiency.

Findings

Reduces likelihood computation in MH for large datasets

Balances bias and variance to improve sampling efficiency

Allows more samples per unit time despite asymptotic bias

Abstract

Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets? We argue that computing the likelihood for N datapoints in the Metropolis-Hastings (MH) test to reach a single binary decision is computationally inefficient. We introduce an approximate MH rule based on a sequential hypothesis test that allows us to accept or reject samples with high confidence using only a fraction of the data required for the exact MH rule. While this method introduces an asymptotic bias, we show that this bias can be controlled and is more than offset by a decrease in variance due to our ability to draw more samples per unit of time.