Transitional annealed adaptive slice sampling for Gaussian process hyper-parameter estimation

TL;DR

This paper introduces a novel Bayesian sampling algorithm combining slice sampling, annealing, and parallelization to improve Gaussian process hyper-parameter estimation, providing more comprehensive uncertainty quantification.

Contribution

It proposes a new Transitional Annealed Adaptive Slice Sampling method that enhances hyper-parameter inference for Gaussian processes over traditional optimization techniques.

Findings

Improved mixing in hyper-parameter sampling.

Enhanced uncertainty quantification in Gaussian process models.

Algorithm demonstrated on Gaussian process hyper-parameter estimation examples.

Abstract

Surrogate models have become ubiquitous in science and engineering for their capability of emulating expensive computer codes, necessary to model and investigate complex phenomena. Bayesian emulators based on Gaussian processes adequately quantify the uncertainty that results from the cost of the original simulator, and thus the inability to evaluate it on the whole input space. However, it is common in the literature that only a partial Bayesian analysis is carried out, whereby the underlying hyper-parameters are estimated via gradient-free optimisation or genetic algorithms, to name a few methods. On the other hand, maximum a posteriori (MAP) estimation could discard important regions of the hyper-parameter space. In this paper, we carry out a more complete Bayesian inference, that combines Slice Sampling with some recently developed Sequential Monte Carlo samplers. The resulting algorithm improves the mixing in the sampling through delayed-rejection, the inclusion of an annealing scheme akin to Asymptotically Independent Markov Sampling and parallelisation via Transitional Markov Chain Monte Carlo. Examples related to the estimation of Gaussian process hyper-parameters are presented. For the purpose of reproducibility, further development, and use in other applications, the code to generate the examples in this paper is freely available for download at http://github.com/agarbuno/ta2s2_codes