Validating Gaussian Process Models with Simulation-Based Calibration

TL;DR

This paper introduces a simulation-based calibration method to validate Gaussian process models, ensuring correct implementation and guiding hyperparameter marginalization, with demonstrated effectiveness in bug detection.

Contribution

The paper presents a novel validation procedure for Gaussian process models that helps verify implementation correctness and determine when hyperparameter marginalization is needed.

Findings

Effective in identifying implementation bugs

Assists in deciding hyperparameter marginalization

Validated through practical application

Abstract

Gaussian process priors are a popular choice for Bayesian analysis of regression problems. However, the implementation of these models can be complex, and ensuring that the implementation is correct can be challenging. In this paper we introduce Gaussian process simulation-based calibration, a procedure for validating the implementation of Gaussian process models and demonstrate the efficacy of this procedure in identifying a bug in existing code. We also present a novel application of this procedure to identify when marginalisation of the model hyperparameters is necessary.