Hess-MC2: Sequential Monte Carlo Squared using Hessian Information and Second Order Proposals

TL;DR

This paper introduces a novel second-order proposal method for Sequential Monte Carlo Squared (SMC$^2$) that leverages Hessian information to improve posterior approximation accuracy and computational efficiency in Bayesian inference.

Contribution

The paper is the first to incorporate second-order (Hessian-based) proposals into the SMC$^2$ framework, enhancing exploration and accuracy over existing methods.

Findings

Second-order proposals improve step-size selection.

Hessian-based proposals enhance posterior approximation accuracy.

Experimental results show benefits over first-order methods.

Abstract

When performing Bayesian inference using Sequential Monte Carlo (SMC) methods, two considerations arise: the accuracy of the posterior approximation and computational efficiency. To address computational demands, Sequential Monte Carlo Squared (SMC$^2$) is well-suited for high-performance computing (HPC) environments. The design of the proposal distribution within SMC$^2$ can improve accuracy and exploration of the posterior as poor proposals may lead to high variance in importance weights and particle degeneracy. The Metropolis-Adjusted Langevin Algorithm (MALA) uses gradient information so that particles preferentially explore regions of higher probability. In this paper, we extend this idea by incorporating second-order information, specifically the Hessian of the log-target. While second-order proposals have been explored previously in particle Markov Chain Monte Carlo (p-MCMC) methods, we are the first to introduce them within the SMC$^2$ framework. Second-order proposals not only use the gradient (first-order derivative), but also the curvature (second-order derivative) of the target distribution. Experimental results on synthetic models highlight the benefits of our approach in terms of step-size selection and posterior approximation accuracy when compared to other proposals.