Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling

TL;DR

This paper introduces a computationally efficient Bayesian sampling method for Gaussian process hyper-parameter estimation, effectively capturing structural and model uncertainties, especially in multimodal distributions, with applications demonstrated through examples.

Contribution

It extends Asymptotically Independent Markov Sampling for Gaussian processes, enabling efficient uncertainty quantification of hyper-parameters in limited data scenarios.

Findings

Efficient sampling of multimodal hyper-parameter distributions.

Structural uncertainty quantification as a by-product of Bayesian inference.

Code availability facilitates reproducibility and further research.

Abstract

Gaussian process emulators of computationally expensive computer codes provide fast statistical approximations to model physical processes. The training of these surrogates depends on the set of design points chosen to run the simulator. Due to computational cost, such training set is bound to be limited and quantifying the resulting uncertainty in the hyper-parameters of the emulator by uni-modal distributions is likely to induce bias. In order to quantify this uncertainty, this paper proposes a computationally efficient sampler based on an extension of Asymptotically Independent Markov Sampling, a recently developed algorithm for Bayesian inference. Structural uncertainty of the emulator is obtained as a by-product of the Bayesian treatment of the hyper-parameters. Additionally, the user can choose to perform stochastic optimisation to sample from a neighbourhood of the Maximum a Posteriori estimate, even in the presence of multimodality. Model uncertainty is also acknowledged through numerical stabilisation measures by including a nugget term in the formulation of the probability model. The efficiency of the proposed sampler is illustrated in examples where multi-modal distributions are encountered. For the purpose of reproducibility, further development, and use in other applications the code used to generate the examples is freely available for download at https://github.com/agarbuno/paims_codes