Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems

TL;DR

This paper develops an infinite-dimensional Stein variational gradient descent method for Bayesian inverse problems, enabling efficient sampling from complex posteriors in large-scale settings.

Contribution

It introduces a rigorous infinite-dimensional framework for SVGD using operator-valued kernels and extends the method with preconditioning for improved efficiency.

Findings

The method accurately samples from posteriors in inverse problems.

Numerical experiments validate theoretical properties and effectiveness.

Potential for large-scale nonlinear inverse problem applications.

Abstract

In this paper, we propose an infinite-dimensional version of the Stein variational gradient descent (iSVGD) method for solving Bayesian inverse problems. The method can generate approximate samples from posteriors efficiently. Based on the concepts of operator-valued kernels and vector-valued reproducing kernel Hilbert spaces, a rigorous definition is given for the infinite-dimensional objects, e.g., the Stein operator, which are proved to be the limit of finite-dimensional ones. Moreover, a more efficient iSVGD with preconditioning operators is constructed by generalizing the change of variables formula and introducing a regularity parameter. The proposed algorithms are applied to an inverse problem of the steady state Darcy flow equation. Numerical results confirm our theoretical findings and demonstrate the potential applications of the proposed approach in the posterior sampling of large-scale nonlinear statistical inverse problems.