Trustworthy Prediction with Gaussian Process Knowledge Scores

TL;DR

This paper introduces a knowledge score for Gaussian process regression predictions that measures how much data has reduced uncertainty, helping to assess prediction reliability in various tasks.

Contribution

It proposes an interpretable, bounded knowledge score for GPR predictions, enhancing the understanding of prediction confidence and reliability.

Findings

Knowledge score predicts prediction accuracy effectively.

Improves anomaly detection, extrapolation, and data imputation.

Enables better assessment of model uncertainty.

Abstract

Probabilistic models are often used to make predictions in regions of the data space where no observations are available, but it is not always clear whether such predictions are well-informed by previously seen data. In this paper, we propose a knowledge score for predictions from Gaussian process regression (GPR) models that quantifies the extent to which observing data have reduced our uncertainty about a prediction. The knowledge score is interpretable and naturally bounded between 0 and 1. We demonstrate in several experiments that the knowledge score can anticipate when predictions from a GPR model are accurate, and that this anticipation improves performance in tasks such as anomaly detection, extrapolation, and missing data imputation. Source code for this project is available online at https://github.com/KurtButler/GP-knowledge.