Multivariate emulation of computer simulators: model selection and diagnostics with application to a humanitarian relief model

TL;DR

This paper develops a unified Bayesian framework for multivariate emulator models, introduces diagnostics for their assessment, and compares parametric and nonparametric approaches using a humanitarian relief simulator.

Contribution

It introduces novel diagnostics and techniques for assessing multivariate emulators, and compares parametric and nonparametric methods in a practical humanitarian application.

Findings

Nonparametric emulators show comparable prediction accuracy to parametric ones.

Diagnostics effectively identify emulator inadequacies.

Sensitivity analysis reveals key input variables affecting outputs.

Abstract

We present a common framework for Bayesian emulation methodologies for multivariate-output simulators, or computer models, that employ either parametric linear models or nonparametric Gaussian processes. Novel diagnostics suitable for multivariate covariance-separable emulators are developed and techniques to improve the adequacy of an emulator are discussed and implemented. A variety of emulators are compared for a humanitarian relief simulator, modelling aid missions to Sicily after a volcanic eruption and earthquake, and a sensitivity analysis is conducted to determine the sensitivity of the simulator output to changes in the input variables. The results from parametric and nonparametric emulators are compared in terms of prediction accuracy, uncertainty quantification and scientific interpretability.