A Framework for Evaluating Approximation Methods for Gaussian Process Regression

TL;DR

This paper evaluates various approximation methods for Gaussian process regression, providing a framework to compare their accuracy and efficiency across different datasets and encouraging standardized benchmarking.

Contribution

It introduces a systematic evaluation framework for GPR approximation methods and empirically compares four algorithms on multiple prediction tasks.

Findings

Different approximation methods vary in accuracy and computational efficiency.

Assessment based on prediction quality versus compute time is effective.

Code availability promotes reproducibility and future benchmarking.

Abstract

Gaussian process (GP) predictors are an important component of many Bayesian approaches to machine learning. However, even a straightforward implementation of Gaussian process regression (GPR) requires O(n^2) space and O(n^3) time for a dataset of n examples. Several approximation methods have been proposed, but there is a lack of understanding of the relative merits of the different approximations, and in what situations they are most useful. We recommend assessing the quality of the predictions obtained as a function of the compute time taken, and comparing to standard baselines (e.g., Subset of Data and FITC). We empirically investigate four different approximation algorithms on four different prediction problems, and make our code available to encourage future comparisons.