Calibrated Computation-Aware Gaussian Processes

TL;DR

This paper introduces CAGP-GS, a new calibration method for computation-aware Gaussian processes using Gauss-Seidel iterations, improving uncertainty quantification and scalability for large regression problems.

Contribution

It proves calibration conditions for probabilistic linear solvers in Gaussian processes and proposes a new framework, CAGP-GS, that enhances uncertainty estimates and computational efficiency.

Findings

CAGP-GS provides well-calibrated uncertainty quantification.

It performs favorably on low-dimensional test sets with few iterations.

The approach is effective on large-scale temperature regression problems.

Abstract

Gaussian processes are notorious for scaling cubically with the size of the training set, preventing application to very large regression problems. Computation-aware Gaussian processes (CAGPs) tackle this scaling issue by exploiting probabilistic linear solvers to reduce complexity, widening the posterior with additional computational uncertainty due to reduced computation. However, the most commonly used CAGP framework results in (sometimes dramatically) conservative uncertainty quantification, making the posterior unrealistic in practice. In this work, we prove that if the utilised probabilistic linear solver is calibrated, in a rigorous statistical sense, then so too is the induced CAGP. We thus propose a new CAGP framework, CAGP-GS, based on using Gauss-Seidel iterations for the underlying probabilistic linear solver. CAGP-GS performs favourably compared to existing approaches when the test set is low-dimensional and few iterations are performed. We test the calibratedness on a synthetic problem, and compare the performance to existing approaches on a large-scale global temperature regression problem.