Scale estimation and rate-unbiasedness for Gaussian processes under smoothness misspecification

TL;DR

This paper introduces the concept of rate-unbiasedness for Gaussian process regression, showing that scale estimation can ensure correct uncertainty quantification even under smoothness misspecification.

Contribution

It defines rate-unbiasedness as a relaxed optimality criterion and proves that scale estimation suffices to achieve it in common settings.

Findings

Rate-unbiasedness ensures bounded error ratios as data increases.

Scale estimation is sufficient for rate-unbiasedness in many Gaussian process models.

Uncertainty quantification remains accurate under smoothness misspecification.

Abstract

Gaussian process regression is used throughout statistics and machine learning for prediction and uncertainty quantification. A Gaussian process is specified by its mean and covariance functions. Many covariance functions, including Mat\'erns, have a smoothness parameter that is notoriously difficult to specify correctly or estimate from the data. In practice, the smoothness parameter is often selected more or less arbitrarily. We introduce rate-unbiasedness, a relaxed notion of asymptotic optimality which requires that the expected ratio of the mean-square error presumed by a potentially misspecified model and the true, but unknown, mean-square error remain bounded away from zero and infinity as more data are obtained. A rate-unbiased model provides uncertainty quantification that is of correct order of magnitude. We then prove that scale estimation suffices for rate-unbiasedness in a variety of common settings. As estimation of the scale of a Gaussian process is routine and requires no optimisation, rate-unbiasedness can be achieved in many applications.