Ergodicity of Approximate MCMC Chains with Applications to Large Data Sets

TL;DR

This paper establishes quantitative bounds for the ergodicity of approximate MCMC algorithms, analyzing their bias-variance trade-offs, especially in large data contexts, and applies these results to recent algorithm variants.

Contribution

It provides the first rigorous bounds on the ergodicity of approximate MCMC methods and explores their bias-variance characteristics in large data settings.

Findings

Derived quantitative ergodicity bounds for approximate MCMC

Analyzed bias-variance trade-offs in large data scenarios

Applied bounds to recent approximate MCMC algorithms

Abstract

In many modern applications, difficulty in evaluating the posterior density makes performing even a single MCMC step slow. This difficulty can be caused by intractable likelihood functions, but also appears for routine problems with large data sets. Many researchers have responded by running approximate versions of MCMC algorithms. In this note, we develop quantitative bounds for showing the ergodicity of these approximate samplers. We then use these bounds to study the bias-variance trade-off of approximate MCMC algorithms. We apply our results to simple versions of recently proposed algorithms, including a variant of the "austerity" framework of Korratikara et al.