Practical heteroskedastic Gaussian process modeling for large simulation experiments

TL;DR

This paper introduces a computationally efficient Gaussian process modeling approach for large simulation experiments with input-dependent noise, leveraging likelihood-based inference and replication to improve accuracy and speed.

Contribution

It develops a unified likelihood-based Gaussian process framework that efficiently handles heteroskedasticity and replication without approximations, using matrix identities and latent variables.

Findings

Efficient inference for heteroskedastic Gaussian processes using Woodbury identity

Demonstrated improved modeling in manufacturing and epidemic simulations

Achieved rapid optimization with closed-form derivatives

Abstract

We present a unified view of likelihood based Gaussian progress regression for simulation experiments exhibiting input-dependent noise. Replication plays an important role in that context, however previous methods leveraging replicates have either ignored the computational savings that come from such design, or have short-cut full likelihood-based inference to remain tractable. Starting with homoskedastic processes, we show how multiple applications of a well-known Woodbury identity facilitate inference for all parameters under the likelihood (without approximation), bypassing the typical full-data sized calculations. We then borrow a latent-variable idea from machine learning to address heteroskedasticity, adapting it to work within the same thrifty inferential framework, thereby simultaneously leveraging the computational and statistical efficiency of designs with replication. The result is an inferential scheme that can be characterized as single objective function, complete with closed form derivatives, for rapid library-based optimization. Illustrations are provided, including real-world simulation experiments from manufacturing and the management of epidemics.