Accelerated Markov Chain Monte Carlo Algorithms on Discrete States

TL;DR

This paper introduces an accelerated Markov Chain Monte Carlo algorithm for discrete states, leveraging Nesterov's method and Wasserstein geometry to improve sampling efficiency and estimate score functions without normalization.

Contribution

It extends classical MCMC with a momentum-based acceleration framework on discrete probability simplices, incorporating Wasserstein geometry and Hamiltonian flows.

Findings

Demonstrates improved sampling efficiency on lattice and hypercube distributions.

Shows effectiveness in estimating discrete score functions without normalization.

Validates the approach with numerical experiments on Gaussian mixtures.

Abstract

We propose a class of discrete state sampling algorithms based on Nesterov's accelerated gradient method, which extends the classical Metropolis-Hastings (MH) algorithm. The evolution of the discrete states probability distribution governed by MH can be interpreted as a gradient descent direction of the Kullback--Leibler (KL) divergence, via a mobility function and a score function. Specifically, this gradient is defined on a probability simplex equipped with a discrete Wasserstein-2 metric with a mobility function. This motivates us to study a momentum-based acceleration framework using damped Hamiltonian flows on the simplex set, whose stationary distribution matches the discrete target distribution. Furthermore, we design an interacting particle system to approximate the proposed accelerated sampling dynamics. The extension of the algorithm with a general choice of potentials and mobilities is also discussed. In particular, we choose the accelerated gradient flow of the relative Fisher information, demonstrating the advantages of the algorithm in estimating discrete score functions without requiring the normalizing constant and keeping positive probabilities. Numerical examples, including sampling on a Gaussian mixture supported on lattices or a distribution on a hypercube, demonstrate the effectiveness of the proposed discrete-state sampling algorithm.