Annealed Leap-Point Sampler for Multimodal Target Distributions

TL;DR

The paper introduces ALPS, a novel MCMC method that efficiently explores high-dimensional multimodal distributions by combining annealing, mode search, and Laplace approximation, improving convergence and scalability.

Contribution

ALPS is a new sampling algorithm that automates mode jumping in multimodal distributions using annealed densities and Laplace approximations, with theoretical complexity guarantees.

Findings

ALPS effectively explores complex multimodal distributions in real-world applications.

Theoretical analysis shows ALPS's convergence scales linearly with dimension.

ALPS demonstrates improved efficiency over traditional MCMC methods.

Abstract

In Bayesian statistics, exploring high-dimensional multimodal posterior distributions poses major challenges for existing MCMC approaches. This paper introduces the Annealed Leap-Point Sampler (ALPS), which augments the target distribution state space with modified annealed (cooled) distributions, in contrast to traditional tempering approaches. The coldest state is chosen such that its annealed density is well-approximated locally by a Laplace approximation. This allows for automated setup of a scalable mode-leaping independence sampler. ALPS requires an exploration component to search for the mode locations, which can either be run adaptively in parallel to improve these mode-jumping proposals, or else as a pre-computation step. A theoretical analysis shows that for a d-dimensional problem the coolest temperature level required only needs to be linear in dimension, $\mathcal{O}\left(d\right)$, implying that the number of iterations needed for ALPS to converge is $\mathcal{O}\left(d\right)$ (typically leading to overall complexity $\mathcal{O}\left(d^3\right)$ when computational cost per iteration is taken into account). ALPS is illustrated on several complex, multimodal distributions that arise from real-world applications. This includes a seemingly-unrelated regression (SUR) model of longitudinal data from U.S. manufacturing firms, as well as a spectral density model that is used in analytical chemistry for identification of molecular biomarkers.