Likelihood-informed dimension reduction across tempered Bayesian posteriors

TL;DR

This paper extends likelihood-informed subspace methods to tempered Bayesian posteriors, improving dimension reduction and sampling efficiency in noisy, data-limited scenarios, especially for complex models.

Contribution

It introduces the concept of $oldsymbol{oldsymbol{ ext{}} ext{ extalpha} ext{-LIS}}$, generalizing LIS to tempered distributions, with theoretical guarantees and practical extensions for noisy, derivative-free settings.

Findings

$oldsymbol{ extalpha<1}$ often yields near-optimal spaces.

Accumulated approach outperforms the $oldsymbol{ extalpha=1}$ case in noisy, data-scarce environments.

Method is robust for emulating forward maps in chaotic or stochastic systems.

Abstract

Scientific computer simulations cannot represent all scales in realistic applications. To bridge this model-data gap, parameters are injected into models and constrained with noisy data using Bayesian inversion. To reduce the number of simulator evaluations, which can be 10^5 or more, modern approaches employ dimension reduction in conjunction with emulation of the forward map (that contains the simulator). Due to scarcity of model evaluations and data, this dimension reduction becomes very important for posterior sampling performance. Recent work on likelihood-informed subspaces (LIS) truncates to informative directions by optimizing bounds on information loss, and though mathematically well-adapted to sampling, they are often restrictive in practice. In this work, we provably generalize this methodology to facilitate application to $\alpha$-tempered (i.e., annealed, power-posterior) distributions for $\alpha$ in [0,1]. We provide theory to build partially-informed spaces termed $\alpha$-LIS. We show how $\alpha$ < 1 can often produce near-optimal spaces. In addition, we focus on applying $\alpha$-LIS to practical cases, where the available data is severely limited and noisy. We propose and test extensions for utilizing data from the entire sequence of distributions $\alpha$_0 < ... < $\alpha$_k, and use simple approximations of model gradients so that our approach can be used for emulation of forward maps for chaotic or stochastic systems where derivatives are unavailable or uninformative due to noise. In experiments, our accumulated approach is much more robust to these challenging circumstances than the theoretically optimal $\alpha$ = 1.