Parallelize Single-Site Dynamics up to Dobrushin Criterion

TL;DR

This paper presents a parallel algorithm for simulating single-site Markov chain dynamics under relaxed Dobrushin conditions, enabling efficient RNC sampling for high-dimensional graphical models and related problems.

Contribution

It introduces a generic parallel algorithm that simulates single-site dynamics under a relaxed Dobrushin influence condition, achieving exponential speedup and RNC sampling for various models.

Findings

Parallel algorithm achieves $O(N/n + ext{log} n)$ depth for $N$ steps.

Under the $ ext{ell}_p$-Dobrushin condition, depth reduces to $O( ext{log} N + ext{log} n)$.

Effective RNC samplers for Ising, hardcore, and SAT models within their regimes.

Abstract

Single-site dynamics are canonical Markov chain based algorithms for sampling from high-dimensional distributions, such as the Gibbs distributions of graphical models. We introduce a simple and generic parallel algorithm that faithfully simulates single-site dynamics. Under a much relaxed, asymptotic variant of the $\ell_p$-Dobrushin's condition -- where the Dobrushin's influence matrix has a bounded $\ell_p$-induced operator norm for an arbitrary $p\in[1, \infty]$ -- our algorithm simulates $N$ steps of single-site updates within a parallel depth of $O\left({N}/{n}+\log n\right)$ on $\tilde{O}(m)$ processors, where $n$ is the number of sites and $m$ is the size of the graphical model. For Boolean-valued random variables, if the $\ell_p$-Dobrushin's condition holds -- specifically, if the $\ell_p$-induced operator norm of the Dobrushin's influence matrix is less than~$1$ -- the parallel depth can be further reduced to $O(\log N+\log n)$, achieving an exponential speedup. These results suggest that single-site dynamics with near-linear mixing times can be parallelized into $\mathsf{RNC}$ sampling algorithms, independent of the maximum degree of the underlying graphical model, as long as the Dobrushin influence matrix maintains a bounded operator norm. We show the effectiveness of this approach with $\mathsf{RNC}$ samplers for the hardcore and Ising models within their uniqueness regimes, as well as an $\mathsf{RNC}$ SAT sampler for satisfying solutions of CNF formulas in a local lemma regime. Furthermore, by employing non-adaptive simulated annealing, these $\mathsf{RNC}$ samplers can be transformed into $\mathsf{RNC}$ algorithms for approximate counting.