Discrete Hamiltonian-Assisted Metropolis Sampling

TL;DR

This paper introduces Discrete HAMS, a novel gradient-based Markov Chain Monte Carlo method for discrete distributions that incorporates Hamiltonian dynamics, resulting in improved exploration and performance over existing algorithms.

Contribution

The paper proposes Discrete HAMS (DHAMS), a new sampler that combines gradient information, Gaussian momentum, and Hamiltonian sampling techniques for discrete distributions.

Findings

DHAMS achieves generalized detailed balance enabling irreversible exploration.

DHAMS is rejection-free for linear potential functions.

Experiments show DHAMS outperforms existing algorithms on ordinal and binary distributions.

Abstract

Gradient-based Markov Chain Monte Carlo methods have recently received much attention for sampling discrete distributions, with interesting connections to their continuous counterparts. For examples, there are two discrete analogues to the Metropolis-adjusted Langevin Algorithm (MALA). As motivated by Hamiltonian-Assisted Metropolis Sampling (HAMS), we propose Discrete HAMS (DHAMS), a discrete sampler which, for the first time, not only exploits gradient information but also incorporates a Gaussian momentum variable and samples a Hamiltonian as an augmented distribution. DHAMS is derived through several steps, including an auxiliary-variable proposal scheme, negation and gradient correction for the momentum variable, and over-relaxation for the state variable. Two distinctive properties are achieved simultaneously. One is generalized detailed balance, which enables irreversible exploration of the target distribution. The other is a rejection-free property for a target distribution with a linear potential function. In experiments involving both ordinal and binary distributions, DHAMS algorithms consistently yield superior performance compared with existing algorithms.