Modernizing full posterior inference for surrogate modeling of categorical-output simulation experiments

TL;DR

This paper introduces a scalable Bayesian Gaussian process classification method using elliptical slice sampling and Vecchia approximation, improving uncertainty quantification for large-scale simulation surrogate modeling.

Contribution

It presents a novel GP classification framework that enhances posterior inference accuracy and efficiency for large datasets, especially in complex simulation contexts.

Findings

Superiority over variational inference in black hole simulations

Effective for large-scale binary classification tasks

Extended to nonstationary, warped input models

Abstract

Gaussian processes (GPs) are powerful tools for nonlinear classification in which latent GPs are combined with link functions. But GPs do not scale well to large training data. This is compounded for classification where the latent GPs require Markov chain Monte Carlo integration. Consequently, fully Bayesian, sampling-based approaches had been largely abandoned. Instead, maximization-based alternatives, such as Laplace/variational inference (VI) combined with low rank approximations, are preferred. Though feasible for large training data sets, such schemes sacrifice uncertainty quantification and modeling fidelity, two aspects that are important to our work on surrogate modeling of computer simulation experiments. Here we are motivated by a large scale simulation of binary black hole (BBH) formation. We propose an alternative GP classification framework which uses elliptical slice sampling for Bayesian posterior integration and Vecchia approximation for computational thrift. We demonstrate superiority over VI-based alternatives for BBH simulations and other benchmark classification problems. We then extend our setup to warped inputs for "deep" nonstationary classification.