Convergence Rates of Two-Component MCMC Samplers

TL;DR

This paper establishes that in two-component MCMC samplers, deterministic-scan algorithms converge faster than random-scan ones, providing exact quantitative relationships and extending results to related Metropolis-Hastings variants.

Contribution

It provides the first exact quantitative relationship showing deterministic-scan Gibbs samplers converge faster than random-scan, and links their ergodicity properties to Metropolis-Hastings variants.

Findings

Deterministic-scan converges faster than random-scan in two-component Gibbs samplers.

Geometric ergodicity of some Metropolis-Hastings variants implies ergodicity of associated Gibbs samplers.

Established exact relationships between convergence rates of different two-component MCMC algorithms.

Abstract

Component-wise MCMC algorithms, including Gibbs and conditional Metropolis-Hastings samplers, are commonly used for sampling from multivariate probability distributions. A long-standing question regarding Gibbs algorithms is whether a deterministic-scan (systematic-scan) sampler converges faster than its random-scan counterpart. We answer this question when the samplers involve two components by establishing an exact quantitative relationship between the $L^2$ convergence rates of the two samplers. The relationship shows that the deterministic-scan sampler converges faster. We also establish qualitative relations among the convergence rates of two-component Gibbs samplers and some conditional Metropolis-Hastings variants. For instance, it is shown that if some two-component conditional Metropolis-Hastings samplers are geometrically ergodic, then so are the associated Gibbs samplers.