Rapid mixing in unimodal landscapes and efficient simulatedannealing for multimodal distributions

TL;DR

This paper analyzes the mixing times of certain random walks on high-dimensional grids, showing rapid mixing for unimodal functions and efficient simulated annealing for multimodal distributions, with applications to models like Potts and Bayesian posteriors.

Contribution

It provides theoretical bounds on mixing times for a class of weighted random walks and introduces an annealing scheme that efficiently samples from multimodal distributions.

Findings

Unimodal functions lead to $O(n\,\log n)$ mixing time.

Multimodal functions require exponential mixing time, but annealing reduces it to $O(n^2)$ or $O(n\log n)$.

The methods extend to general graphs using conductance-based schemes.

Abstract

We consider nearest neighbor weighted random walks on the $d$-dimensional box $[n]^d$ that are governed by some function $g:[0,1] \ra [0,\iy)$, by which we mean that standing at $x$, a neighbor $y$ of $x$ is picked at random and the walk then moves there with probability $(1/2)g(n^{-1}y)/(g(n^{-1}y)+g(n^{-1}x))$. We do this for $g$ of the form $f^{m_n}$ for some function $f$ which assumed to be analytically well-behaved and where $m_n \ra \iy$ as $n \ra \iy$. This class of walks covers an abundance of interesting special cases, e.g., the mean-field Potts model, posterior collapsed Gibbs sampling for Latent Dirichlet allocation and certain Bayesian posteriors for models in nuclear physics. The following are among the results of this paper: \begin{itemize} \item If $f$ is unimodal with negative definite Hessian at its global maximum, then the mixing time of the random walk is $O(n\log n)$. \item If $f$ is multimodal, then the mixing time is exponential in $n$, but we show that there is a simulated annealing scheme governed by $f^K$ for an increasing sequence of $K$ that mixes in time $O(n^2)$. Using a varying step size that decreases with $K$, this can be taken down to $O(n\log n)$. \item If the process is studied on a general graph rather than the $d$-dimensional box, a simulated annealing scheme expressed in terms of conductances of the underlying network, works similarly. \end{itemize} Several examples are given, including the ones mentioned above.