Improved MCMC with active subspaces

TL;DR

This paper improves MCMC sampling by introducing particle and Gibbs-based methods that effectively utilize active subspaces, especially when linearity assumptions are violated, reducing computational costs.

Contribution

It proposes novel particle marginal and Gibbs-based MCMC algorithms that enhance active subspace methods for more efficient Bayesian inference.

Findings

Particle marginal MCMC outperforms linearity-based approaches when assumptions are violated.

Gibbs-based methods reduce computational costs in high-dimensional problems.

Active subspace methods can be effectively integrated with particle MCMC techniques.

Abstract

Constantine et al. (2016) introduced a Metropolis-Hastings (MH) approach that target the active subspace of a posterior distribution: a linearly projected subspace that is informed by the likelihood. Schuster et al. (2017) refined this approach to introduce a pseudo-marginal Metropolis-Hastings, integrating out inactive variables through estimating a marginal likelihood at every MH iteration. In this paper we show empirically that the effectiveness of these approaches is limited in the case where the linearity assumption is violated, and suggest a particle marginal Metropolis-Hastings algorithm as an alternative for this situation. Finally, the high computational cost of these approaches leads us to consider alternative approaches to using active subspaces in MCMC that avoid the need to estimate a marginal likelihood: we introduce Metropolis-within-Gibbs and Metropolis-within-particle Gibbs methods that provide a more computationally efficient use of the active subspace.