Deep Gaussian Processes for Multi-fidelity Modeling

TL;DR

This paper introduces a deep Gaussian process model for multi-fidelity data that effectively captures nonlinear correlations and propagates uncertainty, enhancing decision-making in machine learning and scientific applications.

Contribution

It develops a novel multi-fidelity model using deep Gaussian processes with variational inference, avoiding restrictive structural assumptions of previous methods.

Findings

Improves uncertainty quantification in multi-fidelity models

Enhances decision-making accuracy in practical applications

Outperforms existing methods in capturing fidelity correlations

Abstract

Multi-fidelity methods are prominently used when cheaply-obtained, but possibly biased and noisy, observations must be effectively combined with limited or expensive true data in order to construct reliable models. This arises in both fundamental machine learning procedures such as Bayesian optimization, as well as more practical science and engineering applications. In this paper we develop a novel multi-fidelity model which treats layers of a deep Gaussian process as fidelity levels, and uses a variational inference scheme to propagate uncertainty across them. This allows for capturing nonlinear correlations between fidelities with lower risk of overfitting than existing methods exploiting compositional structure, which are conversely burdened by structural assumptions and constraints. We show that the proposed approach makes substantial improvements in quantifying and propagating uncertainty in multi-fidelity set-ups, which in turn improves their effectiveness in decision making pipelines.