Cross Validation and Maximum Likelihood estimations of hyper-parameters of Gaussian processes with model misspecification

TL;DR

This paper compares Maximum Likelihood and Cross Validation methods for estimating Gaussian process hyper-parameters under model misspecification, revealing CV's superiority in misspecified models and ML's optimality when models are correct.

Contribution

It introduces a predictive variance criterion and derives a closed-form expression, providing insights into hyper-parameter estimation under model misspecification.

Findings

CV outperforms ML when the covariance model is misspecified.

ML is optimal when the covariance model is correctly specified.

A new predictive variance criterion is proposed with a closed-form expression.

Abstract

The Maximum Likelihood (ML) and Cross Validation (CV) methods for estimating covariance hyper-parameters are compared, in the context of Kriging with a misspecified covariance structure. A two-step approach is used. First, the case of the estimation of a single variance hyper-parameter is addressed, for which the fixed correlation function is misspecified. A predictive variance based quality criterion is introduced and a closed-form expression of this criterion is derived. It is shown that when the correlation function is misspecified, the CV does better compared to ML, while ML is optimal when the model is well-specified. In the second step, the results of the first step are extended to the case when the hyper-parameters of the correlation function are also estimated from data.