Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation

TL;DR

This paper establishes fundamental limits on the asymptotic behavior of smoothness parameter estimates in Gaussian process interpolation with Matérn kernels, showing they cannot undersmooth the true smoothness and providing bounds for maximum likelihood and cross-validation methods.

Contribution

It proves that the maximum likelihood estimate of the smoothness parameter cannot asymptotically undersmooth the true smoothness in Gaussian process models with Matérn kernels, and characterizes the bounds for different estimation methods.

Findings

MLE cannot undersmooth the true smoothness asymptotically.

MLE recovers the true smoothness for certain functions.

Cross-validation has an asymptotic lower bound that may not be sharp.

Abstract

It is common to model a deterministic response function, such as the output of a computer experiment, as a Gaussian process with a Mat\'ern covariance kernel. The smoothness parameter of a Mat\'ern kernel determines many important properties of the model in the large data limit, including the rate of convergence of the conditional mean to the response function. We prove that the maximum likelihood estimate of the smoothness parameter cannot asymptotically undersmooth the truth when the data are obtained on a fixed bounded subset of $\mathbb{R}^d$. That is, if the data-generating response function has Sobolev smoothness $\nu_0 > d/2$, then the smoothness parameter estimate cannot be asymptotically less than $\nu_0$. The lower bound is sharp. Additionally, we show that maximum likelihood estimation recovers the true smoothness for a class of compactly supported self-similar functions. For cross-validation we prove an asymptotic lower bound $\nu_0 - d/2$, which however is unlikely to be sharp. The results are based on approximation theory in Sobolev spaces and some general theorems that restrict the set of values that the parameter estimators can take.