Residual Gaussian Process: A Tractable Nonparametric Bayesian Emulator for Multi-fidelity Simulations

TL;DR

This paper introduces a residual Gaussian process model for multi-fidelity simulations that efficiently combines different fidelity levels with uncertainty quantification, suitable for high-dimensional problems and active learning.

Contribution

It proposes a novel additive Gaussian process structure with closed-form solutions, enabling scalable and accurate multi-fidelity modeling with uncertainty estimates.

Findings

Effective on univariate benchmarks

Successful application to multivariate problems

Active learning improves model performance with limited data

Abstract

Challenges in multi-fidelity modeling relate to accuracy, uncertainty estimation and high-dimensionality. A novel additive structure is introduced in which the highest fidelity solution is written as a sum of the lowest fidelity solution and residuals between the solutions at successive fidelity levels, with Gaussian process priors placed over the low fidelity solution and each of the residuals. The resulting model is equipped with a closed-form solution for the predictive posterior, making it applicable to advanced, high-dimensional tasks that require uncertainty estimation. Its advantages are demonstrated on univariate benchmarks and on three challenging multivariate problems. It is shown how active learning can be used to enhance the model, especially with a limited computational budget. Furthermore, error bounds are derived for the mean prediction in the univariate case.