Multi-fidelity Hierarchical Neural Processes

TL;DR

This paper introduces MF-HNP, a neural process-based model for multi-fidelity surrogate modeling that effectively combines data from different simulation fidelities, improving scalability and accuracy in high-dimensional complex tasks.

Contribution

MF-HNP is a novel neural latent variable model that overcomes limitations of Gaussian process methods, enabling scalable, flexible multi-fidelity modeling for high-dimensional data.

Findings

Achieves competitive accuracy and uncertainty estimation in epidemiology and climate modeling.

Handles non-nested, high-dimensional data with varying input/output dimensions.

Significantly speeds up complex high-dimensional simulations.

Abstract

Science and engineering fields use computer simulation extensively. These simulations are often run at multiple levels of sophistication to balance accuracy and efficiency. Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. Cheap data generated from low-fidelity simulators can be combined with limited high-quality data generated by an expensive high-fidelity simulator. Existing methods based on Gaussian processes rely on strong assumptions of the kernel functions and can hardly scale to high-dimensional settings. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. MF-HNP inherits the flexibility and scalability of Neural Processes. The latent variables transform the correlations among different fidelity levels from observations to latent space. The predictions across fidelities are conditionally independent given the latent states. It helps alleviate the error propagation issue in existing methods. MF-HNP is flexible enough to handle non-nested high dimensional data at different fidelity levels with varying input and output dimensions. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation. In contrast to deep Gaussian Processes with only low-dimensional (< 10) tasks, our method shows great promise for speeding up high-dimensional complex simulations (over 7000 for epidemiology modeling and 45000 for climate modeling).