Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling

TL;DR

This paper introduces a transfer learning approach for CNN-based surrogate models that efficiently combines multi-dimensional PDE solutions and their lower-dimensional approximations, significantly reducing data costs.

Contribution

It presents a novel transfer learning method that leverages multi-dimensional and lower-dimensional data to improve surrogate modeling efficiency for PDEs.

Findings

Surrogate models outperform Monte Carlo methods in uncertainty quantification.

Transfer learning reduces data requirements for training CNN surrogates.

Approach effectively captures high-dimensional PDE mappings with less data.

Abstract

The development of efficient surrogates for partial differential equations (PDEs) is a critical step towards scalable modeling of complex, multiscale systems-of-systems. Convolutional neural networks (CNNs) have gained popularity as the basis for such surrogate models due to their success in capturing high-dimensional input-output mappings and the negligible cost of a forward pass. However, the high cost of generating training data -- typically via classical numerical solvers -- raises the question of whether these models are worth pursuing over more straightforward alternatives with well-established theoretical foundations, such as Monte Carlo methods. To reduce the cost of data generation, we propose training a CNN surrogate model on a mixture of numerical solutions to both the $d$-dimensional problem and its ($d-1$)-dimensional approximation, taking advantage of the efficiency savings guaranteed by the curse of dimensionality. We demonstrate our approach on a multiphase flow test problem, using transfer learning to train a dense fully-convolutional encoder-decoder CNN on the two classes of data. Numerical results from a sample uncertainty quantification task demonstrate that our surrogate model outperforms Monte Carlo with several times the data generation budget.