Convergence of Gibbs Sampling: Coordinate Hit-and-Run Mixes Fast

TL;DR

This paper proves that the Coordinate Hit-and-Run algorithm converges efficiently for sampling from convex bodies, providing polynomial mixing time bounds, but also shows it performs worse than other methods in some cases.

Contribution

It establishes polynomial mixing time bounds for the Coordinate Hit-and-Run algorithm and compares its efficiency to other sampling methods.

Findings

CHAR has polynomial mixing time in dimension and diameter.

CHAR's conductance is worse than hit-and-run or ball walk in worst cases.

Provides theoretical guarantees for convergence of a widely used sampling method.

Abstract

The Gibbs Sampler is a general method for sampling high-dimensional distributions, dating back to Turchin, 1971. In each step of the Gibbs Sampler, we pick a random coordinate and re-sample that coordinate from the distribution induced by fixing all other coordinates. While it has become widely used over the past half-century, guarantees of efficient convergence have been elusive. We show that for a convex body $K$ in $\mathbb{R}^{n}$ with diameter $D$, the mixing time of the Coordinate Hit-and-Run (CHAR) algorithm on $K$ is polynomial in $n$ and $D$. We also give a lower bound on the conductance of CHAR, showing that it is strictly worse than hit-and-run or the ball walk in the worst case.