Comparing Scale Parameter Estimators for Gaussian Process Interpolation with the Brownian Motion Prior: Leave-One-Out Cross Validation and Maximum Likelihood

TL;DR

This paper compares cross-validation and maximum likelihood methods for estimating the scale parameter in Gaussian process regression with a Brownian motion prior, showing cross-validation's superior calibration in certain settings.

Contribution

It provides a theoretical comparison of the two methods and introduces interior cross validation as a more adaptable approach for kernel parameter estimation.

Findings

Cross-validation yields asymptotically well-calibrated credible intervals for a broader class of functions.

Maximum likelihood estimation is less effective in achieving calibration in the noiseless setting.

Interior cross validation adapts to an even wider class of ground-truth functions.

Abstract

Gaussian process (GP) regression is a Bayesian nonparametric method for regression and interpolation, offering a principled way of quantifying the uncertainties of predicted function values. For the quantified uncertainties to be well-calibrated, however, the kernel of the GP prior has to be carefully selected. In this paper, we theoretically compare two methods for choosing the kernel in GP regression: cross-validation and maximum likelihood estimation. Focusing on the scale-parameter estimation of a Brownian motion kernel in the noiseless setting, we prove that cross-validation can yield asymptotically well-calibrated credible intervals for a broader class of ground-truth functions than maximum likelihood estimation, suggesting an advantage of the former over the latter. Finally, motivated by the findings, we propose interior cross validation, a procedure that adapts to an even broader class of ground-truth functions.