Covariance-Informed Subspace: an Adaptive Gradient-Free Input Dimension Reduction Method for Bayesian Inference

TL;DR

This paper introduces a gradient-free dimension reduction method for Bayesian inference that leverages covariance ratios to identify data-informed directions, especially useful when gradient computation is impractical.

Contribution

It develops a novel covariance-informed subspace method that does not require gradient information, extending likelihood-informed subspace techniques to simulation-based inference.

Findings

Improved posterior approximation in linear Gaussian cases.

Effective dimension reduction demonstrated in groundwater and atmospheric problems.

Method suitable for high-dimensional, gradient-free Bayesian inference.

Abstract

This paper addresses the challenge of dimension reduction (DR) in Bayesian inference of high-resolution two-or three-dimensional fields, where a priori parametrizations require a large number of terms. The underlying idea is common to state-of-the-art methods in which the parameter space is decomposed into two subspaces, one informed by the likelihood and one constrained by the prior. DR techniques generally use gradient information from the log-likelihood to derive the corresponding subspaces. However, the gradient may be unavailable or expensive to compute accurately, for instance in the case of simulation-based inference. Inspired by approaches based on likelihood-informed subspaces, we develop a new DR method tailored for settings where gradient computation is not feasible. More specifically, we propose a gradient-free indicator for determining whether a direction is informed by the data. This indicator is derived from the posterior-to-prior covariance ratio introduced in Spantini et al. (2015). We show that, in the linear Gaussian case, this indicator combined with an approximate likelihood leads to a better posterior approximation. The method is then extended to nonlinear cases, and strategies to approximate the posterior covariance are detailed. We demonstrate the effectiveness of this DR through two high-dimensional inference problems arising from groundwater and atmospheric applications.