An invitation to adaptive Markov chain Monte Carlo convergence theory

TL;DR

This paper presents an accessible martingale-based convergence analysis for adaptive MCMC algorithms with uniformly ergodic Markov transitions, providing practical insights into their theoretical validity.

Contribution

It offers a simplified, self-contained convergence proof for adaptive MCMC using martingale techniques, broadening understanding of their theoretical foundations.

Findings

Martingale decomposition technique applies to adaptive MCMC

Conditions accommodate various adaptation schemes

Provides detailed, accessible proofs

Abstract

Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their validity cannot be ensured by standard Markov chains theory. Several different techniques have been suggested to analyse their theoretical properties, many of which are technically involved. The technical nature of the theory may make the methods unnecessarily unappealing. We discuss one technique -- based on a martingale decomposition -- with uniformly ergodic Markov transitions. We provide an accessible and self-contained treatment in this setting, and give detailed proofs of the results discussed in the paper, which only require basic understanding of martingale theory and general state space Markov chain concepts. We illustrate how our conditions can accomodate different types of adaptation schemes, and can give useful insight to the requirements which ensure their validity.