Scalable Gaussian Processes: Advances in Iterative Methods and Pathwise Conditioning

TL;DR

This paper introduces scalable Gaussian process methods that leverage iterative linear solvers and pathwise conditioning, enabling efficient large-scale data processing on modern hardware.

Contribution

It develops a novel combination of iterative methods and pathwise conditioning to improve Gaussian process scalability and computational efficiency.

Findings

Reduces memory requirements for Gaussian processes.

Enables handling larger datasets with modern hardware.

Expresses computations as linear system solutions.

Abstract

Gaussian processes are a powerful framework for uncertainty-aware function approximation and sequential decision-making. Unfortunately, their classical formulation does not scale gracefully to large amounts of data and modern hardware for massively-parallel computation, prompting many researchers to develop techniques which improve their scalability. This dissertation focuses on the powerful combination of iterative methods and pathwise conditioning to develop methodological contributions which facilitate the use of Gaussian processes in modern large-scale settings. By combining these two techniques synergistically, expensive computations are expressed as solutions to systems of linear equations and obtained by leveraging iterative linear system solvers. This drastically reduces memory requirements, facilitating application to significantly larger amounts of data, and introduces matrix multiplication as the main computational operation, which is ideal for modern hardware.