Probabilistic Linear Solvers: A Unifying View

TL;DR

This paper unifies various probabilistic linear solver methods by establishing general equivalence conditions, connecting them to classical iterative methods, and introducing a probabilistic perspective on preconditioning.

Contribution

It provides a unifying theoretical framework for probabilistic linear solvers, linking them to projection methods and classical iterative algorithms like GMRES.

Findings

Established general conditions for equivalence of probabilistic linear solvers.

Connected probabilistic methods to classical projection and iterative methods.

Introduced a probabilistic interpretation of preconditioning.

Abstract

Several recent works have developed a new, probabilistic interpretation for numerical algorithms solving linear systems in which the solution is inferred in a Bayesian framework, either directly or by inferring the unknown action of the matrix inverse. These approaches have typically focused on replicating the behavior of the conjugate gradient method as a prototypical iterative method. In this work surprisingly general conditions for equivalence of these disparate methods are presented. We also describe connections between probabilistic linear solvers and projection methods for linear systems, providing a probabilistic interpretation of a far more general class of iterative methods. In particular, this provides such an interpretation of the generalised minimum residual method. A probabilistic view of preconditioning is also introduced. These developments unify the literature on probabilistic linear solvers, and provide foundational connections to the literature on iterative solvers for linear systems.