Assessing the performance of correlation-based multi-fidelity neural emulators

TL;DR

This paper evaluates multi-fidelity neural emulators that combine low- and high-fidelity data to improve predictive accuracy in complex modeling tasks, analyzing various architectures and configurations.

Contribution

It systematically investigates the performance of multi-fidelity neural emulators across diverse functions and model configurations, highlighting their advantages over single-fidelity approaches.

Findings

Multi-fidelity emulators outperform single-fidelity models in accuracy.

Different neural network architectures exhibit varying effectiveness.

Combining multiple sources of information enhances predictive performance.

Abstract

Outer loop tasks such as optimization, uncertainty quantification or inference can easily become intractable when the underlying high-fidelity model is computationally expensive. Similarly, data-driven architectures typically require large datasets to perform predictive tasks with sufficient accuracy. A possible approach to mitigate these challenges is the development of multi-fidelity emulators, leveraging potentially biased, inexpensive low-fidelity information while correcting and refining predictions using scarce, accurate high-fidelity data. This study investigates the performance of multi-fidelity neural emulators, neural networks designed to learn the input-to-output mapping by integrating limited high-fidelity data with abundant low-fidelity model solutions. We investigate the performance of such emulators for low and high-dimensional functions, with oscillatory character, in the presence of discontinuities, for collections of models with equal and dissimilar parametrization, and for a possibly large number of potentially corrupted low-fidelity sources. In doing so, we consider a large number of architectural, hyperparameter, and dataset configurations including networks with a different amount of spectral bias (Multi-Layered Perceptron, Siren and Kolmogorov Arnold Network), various mechanisms for coordinate encoding, exact or learnable low-fidelity information, and for varying training dataset size. We further analyze the added value of the multi-fidelity approach by conducting equivalent single-fidelity tests for each case, quantifying the performance gains achieved through fusing multiple sources of information.