Complexity Results for MCMC derived from Quantitative Bounds

TL;DR

This paper introduces a modified drift-and-minorization method to derive tight convergence and complexity bounds for high-dimensional MCMC algorithms, demonstrated through Gibbs sampler analyses.

Contribution

It proposes a new approach using large sets and fitted drift functions to obtain quantitative bounds for high-dimensional MCMC convergence.

Findings

Derived complexity upper bounds for Gibbs samplers.

Showed constant iteration convergence for a Gibbs sampler related to the James--Stein estimator.

Demonstrated the method's potential for broad application in high-dimensional MCMC analysis.

Abstract

This paper considers how to obtain MCMC quantitative convergence bounds which can be translated into tight complexity bounds in high-dimensional {settings}. We propose a modified drift-and-minorization approach, which establishes generalized drift conditions defined in subsets of the state space. The subsets are called the "large sets", and are chosen to rule out some "bad" states which have poor drift property when the dimension of the state space gets large. Using the "large sets" together with a "fitted family of drift functions", a quantitative bound can be obtained which can be translated into a tight complexity bound. As a demonstration, we analyze several Gibbs samplers and obtain complexity upper bounds for the mixing time. In particular, for one example of Gibbs sampler which is related to the James--Stein estimator, we show that the number of iterations required for the Gibbs sampler to converge is constant under certain conditions on the observed data and the initial state. It is our hope that this modified drift-and-minorization approach can be employed in many other specific examples to obtain complexity bounds for high-dimensional Markov chains.