Statistical Properties of the Probabilistic Numeric Linear Solver BayesCG

TL;DR

This paper investigates the statistical calibration of BayesCG, a probabilistic extension of the Conjugate Gradient method, proposing new test statistics and demonstrating its favorable properties under certain conditions.

Contribution

It introduces two new test statistics for calibration assessment and analyzes BayesCG's calibration properties, showing it is nearly calibrated and computationally competitive.

Findings

BayesCG is slightly optimistic in its posterior covariances.

Under low-rank Krylov posteriors, BayesCG exhibits desirable calibration properties.

BayesCG is computationally competitive with traditional CG.

Abstract

We analyse the calibration of BayesCG under the Krylov prior, a probabilistic numeric extension of the Conjugate Gradient (CG) method for solving systems of linear equations with symmetric positive definite coefficient matrix. Calibration refers to the statistical quality of the posterior covariances produced by a solver. Since BayesCG is not calibrated in the strict existing notion, we propose instead two test statistics that are necessary but not sufficient for calibration: the Z-statistic and the new S-statistic. We show analytically and experimentally that under low-rank approximate Krylov posteriors, BayesCG exhibits desirable properties of a calibrated solver, is only slightly optimistic, and is computationally competitive with CG.