Multi-fidelity Gaussian process regression for noisy outputs and non-nested experimental designs: a comparison between the recursive and non-recursive formulations

TL;DR

This paper compares recursive and non-recursive multi-fidelity Gaussian process regression methods for noisy, non-nested data, proposing a recursive approach with EM optimization that reduces training time without sacrificing accuracy.

Contribution

It introduces a recursive formulation with a decoupled EM-based optimization for multi-fidelity GPR, demonstrating efficiency gains over classical methods.

Findings

Recursive approach reduces training time significantly.

Maintains competitive predictive accuracy and uncertainty estimates.

Performs well across complex benchmark applications.

Abstract

This paper investigates a recursive formulation of auto-regressive multi-fidelity Gaussian process regression in the challenging setting of noisy and non-nested high- and low-fidelity data. We propose a decoupled optimization strategy based on the expectation-maximization algorithm, which exploits the structure of the recursive model. In particular, we derive closed-form update formulas when the scaling factor is modeled as a parametric linear predictor. This approach is compared with the fully coupled likelihood maximization of the classical non-recursive formulation introduced by Kennedy and O'Hagan. A series of benchmark experiments, covering applications of increasing complexity, highlights the performance of both approaches. The results demonstrate that the proposed recursive strategy significantly reduces training time, especially when large low-fidelity datasets are available, while maintaining competitive predictive accuracy and uncertainty estimation.