Fast Conditional Mixing of MCMC Algorithms for Non-log-concave Distributions

TL;DR

This paper demonstrates that under certain conditions, MCMC algorithms can achieve fast mixing on subsets of the state space even when global mixing is slow, improving sampling efficiency for complex distributions.

Contribution

It introduces a framework for conditional mixing analysis of MCMC, showing fast convergence on subsets where Poincaré inequalities hold, applicable to Gaussian mixtures and Gibbs sampling.

Findings

Conditional mixing can be fast even when global mixing is slow.

Poincaré inequalities on subsets ensure rapid convergence.

Implications for Gaussian mixtures and Gibbs sampling.

Abstract

MCMC algorithms offer empirically efficient tools for sampling from a target distribution $\pi(x) \propto \exp(-V(x))$. However, on the theory side, MCMC algorithms suffer from slow mixing rate when $\pi(x)$ is non-log-concave. Our work examines this gap and shows that when Poincar\'e-style inequality holds on a subset $\mathcal{X}$ of the state space, the conditional distribution of MCMC iterates over $\mathcal{X}$ mixes fast to the true conditional distribution. This fast mixing guarantee can hold in cases when global mixing is provably slow. We formalize the statement and quantify the conditional mixing rate. We further show that conditional mixing can have interesting implications for sampling from mixtures of Gaussians, parameter estimation for Gaussian mixture models and Gibbs-sampling with well-connected local minima.