Preconditioned Langevin Dynamics with Score-Based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems

TL;DR

This paper develops a rigorous infinite-dimensional framework for Langevin dynamics with score-based models in Bayesian inverse problems, providing error estimates, convergence guarantees, and optimal preconditioning strategies.

Contribution

It introduces a novel infinite-dimensional analysis of Langevin dynamics with SGMs, including error bounds, convergence conditions, and optimal preconditioning for stability.

Findings

Error estimates depend explicitly on score approximation error.

Sufficient conditions for global convergence in Kullback-Leibler divergence.

Existence and form of an optimal preconditioner for stability.

Abstract

Designing algorithms for solving high-dimensional Bayesian inverse problems directly in infinite-dimensional function spaces - where such problems are naturally formulated - is crucial to ensure stability and convergence as the discretization of the underlying problem is refined. In this paper, we contribute to this line of work by analyzing a widely used sampler for linear inverse problems: Langevin dynamics driven by score-based generative models (SGMs) acting as priors, formulated directly in function space. Building on the theoretical framework for SGMs in Hilbert spaces, we give a rigorous definition of this sampler in the infinite-dimensional setting and derive, for the first time, error estimates that explicitly depend on the approximation error of the score. As a consequence, we obtain sufficient conditions for global convergence in Kullback-Leibler divergence on the underlying function space. Preventing numerical instabilities requires preconditioning of the Langevin algorithm and we prove the existence and the form of an optimal preconditioner. The preconditioner depends on both the score error and the forward operator and guarantees a uniform convergence rate across all posterior modes. Our analysis applies to both Gaussian and a general class of non-Gaussian priors. Finally, we present examples that illustrate and validate our theoretical findings.