Optimised Annealed Sequential Monte Carlo Samplers

TL;DR

This paper introduces Optimised Annealed SMC (OASMC), a novel adaptive sampling method that improves efficiency, predictability, and GPU performance by leveraging a theoretical performance model and surrogate objectives for schedule adaptation.

Contribution

The paper develops a new adaptive SMC framework based on a performance model, leading to predictable, efficient algorithms like OASMC and OAIS with reduced communication and enhanced GPU performance.

Findings

Up to 100-fold speed improvement over existing methods

Predictable algorithms with deterministic estimate times

Efficient schedule adaptation with constant memory cost

Abstract

Annealed Sequential Monte Carlo (ASMC) samplers are special cases of SMC samplers where the sequence of distributions can be embedded in a smooth path of distributions. Using this underlying path and a performance model based on the variance of the normalising constant estimator, we systematically study dense-schedule limits. From our theory emerges a notion of global barrier, capturing the inherent complexity of normalising constant approximation under our performance model. We then turn the resulting approximations into surrogate objective functions of algorithm performance, using them to guide method development. This leads to novel adaptive methods, Optimised Annealed SMC (OASMC), which address practical difficulties inherent in previous adaptive SMC methods. First, our OASMC algorithms are predictable: they produce a sequence of increasingly precise estimates at deterministic, known times. Second, Optimised Annealed Importance Sampling (OAIS), a special case of OASMC, enables schedule adaptation at a memory cost constant in the number of particles, requiring significantly less communication. Finally, these characteristics make OAIS highly efficient on GPUs. We provide an open-source, high-performance GPU implementation of our method and demonstrate up to a hundred-fold speed improvement compared to state-of-the-art adaptive AIS methods.