Upper and lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo

TL;DR

This paper establishes fundamental lower bounds on the convergence rates of adaptive MCMC algorithms and explores conditions under which these bounds can be nearly achieved, informing optimal adaptation strategies.

Contribution

It provides the first general lower bounds on subgeometric convergence for adaptive MCMC and compares them with upper bounds under fast diminishing adaptation.

Findings

Lower bounds on total variation and weak convergence rates established.

Upper bounds shown to match lower bounds under certain adaptation decay conditions.

Applications demonstrated on Langevin and Metropolis-Hastings algorithms.

Abstract

We investigate lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo under any adaptation strategy. In particular, we prove general lower bounds in total variation and on the weak convergence rate under general adaptation plans. If the adaptation diminishes sufficiently fast, we also develop comparable convergence rate upper bounds that are capable of approximately matching the convergence rate in the subgeometric lower bound. These results provide insight into the optimal design of adaptation strategies and also limitations on the convergence behavior of adaptive Markov chain Monte Carlo. Applications to an adaptive unadjusted Langevin algorithm as well as adaptive Metropolis-Hastings with independent proposals and random-walk proposals are explored.