Auxiliary-variable Exact Hamiltonian Monte Carlo Samplers for Binary Distributions

TL;DR

This paper introduces an exact Hamiltonian Monte Carlo method for binary distributions that outperforms traditional samplers, especially in complex models like spike-and-slab priors and truncated parameters.

Contribution

The paper proposes a novel exact HMC algorithm for binary distributions and extends it to mixed distributions, improving sampling efficiency over existing methods.

Findings

Outperforms Metropolis and Gibbs samplers in several examples

Effective for spike-and-slab priors and truncated parameters

Demonstrates advantages in linear and probit regression models

Abstract

We present a new approach to sample from generic binary distributions, based on an exact Hamiltonian Monte Carlo algorithm applied to a piecewise continuous augmentation of the binary distribution of interest. An extension of this idea to distributions over mixtures of binary and possibly-truncated Gaussian or exponential variables allows us to sample from posteriors of linear and probit regression models with spike-and-slab priors and truncated parameters. We illustrate the advantages of these algorithms in several examples in which they outperform the Metropolis or Gibbs samplers.