Barely Biased Learning for Gaussian Process Regression

TL;DR

This paper proposes an adaptive method for Gaussian process regression that controls bias in estimating the log marginal likelihood, balancing computational effort and accuracy.

Contribution

It introduces a novel adaptive approach to select computation levels for unbiased log marginal likelihood estimation in Gaussian processes.

Findings

Method guarantees small bias in estimation.

Current implementation is less computationally efficient.

Potential for improved bias-variance trade-offs.

Abstract

Recent work in scalable approximate Gaussian process regression has discussed a bias-variance-computation trade-off when estimating the log marginal likelihood. We suggest a method that adaptively selects the amount of computation to use when estimating the log marginal likelihood so that the bias of the objective function is guaranteed to be small. While simple in principle, our current implementation of the method is not competitive computationally with existing approximations.