Quasi-Newton Sequential Monte Carlo

TL;DR

This paper introduces an accelerated Sequential Monte Carlo method that leverages L-BFGS Hessian approximations to improve efficiency and scalability in high-dimensional Bayesian inference tasks.

Contribution

It integrates L-BFGS Hessian approximation into SMC samplers, enabling faster, adaptive, and parallelizable inference without extra gradient evaluations.

Findings

Significantly faster convergence in high-dimensional problems

Effective handling of multi-modal posterior distributions

Maintains unbiased estimation of the normalising constant

Abstract

Sequential Monte Carlo samplers represent a compelling approach to posterior inference in Bayesian models, due to being parallelisable and providing an unbiased estimate of the posterior normalising constant. In this work, we significantly accelerate sequential Monte Carlo samplers by adopting the L-BFGS Hessian approximation which represents the state-of-the-art in full-batch optimisation techniques. The L-BFGS Hessian approximation has only linear complexity in the parameter dimension and requires no additional posterior or gradient evaluations. The resulting sequential Monte Carlo algorithm is adaptive, parallelisable and well-suited to high-dimensional and multi-modal settings, which we demonstrate in numerical experiments on challenging posterior distributions.