Weighted Leave-One-Out Cross Validation

TL;DR

This paper introduces a weighted leave-one-out cross-validation method for more accurate estimation of the integrated squared error in Gaussian process regression, improving model selection accuracy.

Contribution

It proposes a novel weighted LOO approach for ISE estimation that enhances precision over traditional methods, with theoretical and numerical validation.

Findings

Weighted LOO significantly improves ISE estimation accuracy.

Method demonstrates robustness to different Gaussian Process kernels.

Application to model selection shows practical benefits.

Abstract

We present a weighted version of Leave-One-Out (LOO) cross-validation for estimating the Integrated Squared Error (ISE) when approximating an unknown function by a predictor that depends linearly on evaluations of the function over a finite collection of sites. The method relies on the construction of the best linear estimator of the squared prediction error at an arbitrary unsampled site based on squared LOO residuals, assuming that the function is a realization of a Gaussian Process (GP). A theoretical analysis of performance of the ISE estimator is presented, and robustness with respect to the choice of the GP kernel is investigated first analytically, then through numerical examples. Overall, the estimation of ISE is significantly more precise than with classical, unweighted, LOO cross validation. Application to model selection is briefly considered through examples.