Diagnostics for Stochastic Gaussian Process Emulators

TL;DR

This paper develops diagnostic tools for validating stochastic Gaussian process emulators, focusing on independently assessing mean and variance predictions to ensure reliable surrogate models for computationally expensive stochastic simulators.

Contribution

It introduces a novel framework for diagnosing stochastic emulator problems, addressing limitations of existing validation methods for deterministic models.

Findings

Diagnostics effectively identify issues in stochastic emulators

Framework improves emulator validation accuracy

Case study demonstrates practical utility

Abstract

Computer models, also known as simulators, can be computationally expensive to run, and for this reason statistical surrogates, known as emulators, are often used. Any statistical model, including an emulator, should be validated before being used, otherwise resulting decisions can be misguided. We discuss how current methods for validating Gaussian process emulators of deterministic models are insufficient for emulators of stochastic computer models and develop a framework for diagnosing problems in stochastic emulators. These diagnostics are based on independently validating the mean and variance predictions using out-of-sample, replicated, simulator runs. We then also use a building performance simulator as a case study example.