Quantifying Model Uncertainty of Neural Network-based Turbulence Closures

TL;DR

This paper compares three methods for quantifying uncertainty in neural network-based turbulence closures used in CFD, highlighting their relative strengths and weaknesses for nuclear engineering applications.

Contribution

It evaluates Deep Ensembles, Monte-Carlo Dropout, and SVI for uncertainty quantification in NN turbulence models, providing insights into their performance and potential extensions.

Findings

Deep Ensembles achieve the lowest RMSE of 4.31e-4.

Each UQ method produces distinct epistemic uncertainty estimates.

Deep Ensembles tend to be overconfident, MC-Dropout under-confident, SVI offers principled uncertainty.

Abstract

With increasing computational demand, Neural-Network (NN) based models are being developed as pre-trained surrogates for different thermohydraulics phenomena. An area where this approach has shown promise is in developing higher-fidelity turbulence closures for computational fluid dynamics (CFD) simulations. The primary bottleneck to the widespread adaptation of these NN-based closures for nuclear-engineering applications is the uncertainties associated with them. The current paper illustrates three commonly used methods that can be used to quantify model uncertainty in NN-based turbulence closures. The NN model used for the current study is trained on data from an algebraic turbulence closure model. The uncertainty quantification (UQ) methods explored are Deep Ensembles, Monte-Carlo Dropout, and Stochastic Variational Inference (SVI). The paper ends with a discussion on the relative performance of the three methods for quantifying epistemic uncertainties of NN-based turbulence closures, and potentially how they could be further extended to quantify out-of-training uncertainties. For accuracy in turbulence modeling, paper finds Deep Ensembles have the best prediction accuracy with an RMSE of $4.31\cdot10^{-4}$ on the testing inputs followed by Monte-Carlo Dropout and Stochastic Variational Inference. For uncertainty quantification, this paper finds each method produces unique Epistemic uncertainty estimates with Deep Ensembles being overconfident in regions, MC-Dropout being under-confident, and SVI producing principled uncertainty at the cost of function diversity.