Limit theorems for some adaptive MCMC algorithms with subgeometric kernels

TL;DR

This paper establishes ergodicity and strong law of large numbers for a broad class of adaptive MCMC algorithms with subgeometric kernels, including detailed analysis of the Adaptive Metropolis Algorithm under sub-exponential tails.

Contribution

It extends theoretical guarantees to adaptive MCMC algorithms with subgeometric kernels using new drift conditions and diminishing adaptation assumptions.

Findings

Ergodicity holds under diminishing adaptation and polynomial drift conditions.

Strong law of large numbers applies to unbounded functions with polynomial drift.

Adaptive Metropolis Algorithm is analyzed for sub-exponential tail distributions.

Abstract

This paper deals with the ergodicity and the existence of a strong law of large numbers for adaptive Markov Chain Monte Carlo. We show that a diminishing adaptation assumption together with a drift condition for positive recurrence is enough to imply ergodicity. Strengthening the drift condition to a polynomial drift condition yields a strong law of large numbers for possibly unbounded functions. These results broaden considerably the class of adaptive MCMC algorithms for which rigorous analysis is now possible. As an example, we give a detailed analysis of the Adaptive Metropolis Algorithm of Haario et al. (2001) when the target distribution is sub-exponential in the tails.