Sequential Monte Carlo with active subspaces

TL;DR

This paper develops sequential Monte Carlo methods that leverage active subspaces to improve Bayesian inference efficiency in high-dimensional models, especially when likelihoods are uninformative in certain parameter subspaces.

Contribution

It introduces SMC algorithms utilizing active subspaces, including adaptive learning of the subspace and an SMC$^{2}$ variant for enhanced robustness over linear assumptions.

Findings

Demonstrates improved efficiency in high-dimensional Bayesian inference.

Provides adaptive methods for identifying active subspaces.

Shows robustness of SMC$^{2}$ approach in practical scenarios.

Abstract

Monte Carlo methods, such as Markov chain Monte Carlo (MCMC), remain the most regularly-used approach for implementing Bayesian inference. However, the computational cost of these approaches usually scales worse than linearly with the dimension of the parameter space, limiting their use for models with a large number of parameters. However, it is not uncommon for models to have identifiability problems. In this case, the likelihood is not informative about some subspaces of the parameter space, and hence the model effectively has a dimension that is lower than the number of parameters. Constantine et al. (2016) and Schuster et al. (2017) introduced the concept of directly using a Metropolis-Hastings (MH) MCMC on the subspaces of the parameter space that are informed by the likelihood as a means to reduce the dimension of the parameter space that needs to be explored. The same paper introduces an approach for identifying such a subspace in the case where it is a linear transformation of the parameter space: this subspace is known as the active subspace. This paper introduces sequential Monte Carlo (SMC) methods that make use of an active subspace. As well as an SMC counterpart to MH approach of Schuster et al. (2017), we introduce an approach to learn the active subspace adaptively, and an SMC$^{2}$ approach that is more robust to the linearity assumptions made when using active subspaces.