Robust Prediction Interval estimation for Gaussian Processes by Cross-Validation method

TL;DR

This paper introduces a robust two-step method for calibrating Gaussian Process Prediction Intervals, optimizing coverage and width by combining cross-validation, coverage probability adjustment, and Wasserstein distance minimization.

Contribution

It proposes a novel calibration approach for Gaussian Process Prediction Intervals that improves coverage accuracy and interval width using a combination of cross-validation, coverage assessment, and Wasserstein distance.

Findings

Achieves accurate coverage probabilities in prediction intervals.

Produces narrower intervals with maintained coverage.

Outperforms standard methods in calibration accuracy.

Abstract

Probabilistic regression models typically use the Maximum Likelihood Estimation or Cross-Validation to fit parameters. These methods can give an advantage to the solutions that fit observations on average, but they do not pay attention to the coverage and the width of Prediction Intervals. A robust two-step approach is used to address the problem of adjusting and calibrating Prediction Intervals for Gaussian Processes Regression. First, the covariance hyperparameters are determined by a standard Cross-Validation or Maximum Likelihood Estimation method. A Leave-One-Out Coverage Probability is introduced as a metric to adjust the covariance hyperparameters and assess the optimal type II Coverage Probability to a nominal level. Then a relaxation method is applied to choose the hyperparameters that minimize the Wasserstein distance between the Gaussian distribution with the initial hyperparameters (obtained by Cross-Validation or Maximum Likelihood Estimation) and the proposed Gaussian distribution with the hyperparameters that achieve the desired Coverage Probability. The method gives Prediction Intervals with appropriate coverage probabilities and small widths.