Adaptive Cholesky Gaussian Processes

TL;DR

This paper introduces an adaptive method for Gaussian process regression that efficiently selects data subsets during inference, reducing computational costs while maintaining accuracy by leveraging probabilistic bounds and Cholesky decomposition insights.

Contribution

The novel approach adaptively determines subset size during inference using probabilistic bounds, enabling efficient large-scale Gaussian process regression with minimal overhead.

Findings

The method effectively identifies redundant data in large datasets.

It provides probabilistic bounds for model evidence based on subset selection.

The approach reduces computational complexity in Gaussian process models.

Abstract

We present a method to approximate Gaussian process regression models for large datasets by considering only a subset of the data. Our approach is novel in that the size of the subset is selected on the fly during exact inference with little computational overhead. From an empirical observation that the log-marginal likelihood often exhibits a linear trend once a sufficient subset of a dataset has been observed, we conclude that many large datasets contain redundant information that only slightly affects the posterior. Based on this, we provide probabilistic bounds on the full model evidence that can identify such subsets. Remarkably, these bounds are largely composed of terms that appear in intermediate steps of the standard Cholesky decomposition, allowing us to modify the algorithm to adaptively stop the decomposition once enough data have been observed.