Fast Doubly-Adaptive MCMC to Estimate the Gibbs Partition Function with Weak Mixing Time Bounds

TL;DR

This paper introduces a doubly adaptive MCMC method that efficiently estimates Gibbs distribution partition functions, outperforming existing algorithms especially in high-precision scenarios by reducing complexity and sensitivity to mixing time bounds.

Contribution

The paper presents a novel doubly adaptive approach combining adaptive cooling schedules with MCMC mean estimators, improving efficiency and robustness in partition function estimation.

Findings

Reduced computational complexity compared to state-of-the-art methods.

Less sensitivity to loose mixing time bounds.

Significant improvements in high-precision estimation regimes.

Abstract

We present a novel method for reducing the computational complexity of rigorously estimating the partition functions (normalizing constants) of Gibbs (Boltzmann) distributions, which arise ubiquitously in probabilistic graphical models. A major obstacle to practical applications of Gibbs distributions is the need to estimate their partition functions. The state of the art in addressing this problem is multi-stage algorithms, which consist of a cooling schedule, and a mean estimator in each step of the schedule. While the cooling schedule in these algorithms is adaptive, the mean estimation computations use MCMC as a black-box to draw approximate samples. We develop a doubly adaptive approach, combining the adaptive cooling schedule with an adaptive MCMC mean estimator, whose number of Markov chain steps adapts dynamically to the underlying chain. Through rigorous theoretical analysis, we prove that our method outperforms the state of the art algorithms in several factors: (1) The computational complexity of our method is smaller; (2) Our method is less sensitive to loose bounds on mixing times, an inherent component in these algorithms; and (3) The improvement obtained by our method is particularly significant in the most challenging regime of high-precision estimation. We demonstrate the advantage of our method in experiments run on classic factor graphs, such as voting models and Ising models.