Epistemic Modeling Uncertainty of Rapid Neural Network Ensembles for Adaptive Learning

TL;DR

This paper introduces a rapid neural network ensemble method for efficient epistemic uncertainty estimation in aerospace design, significantly reducing training costs while maintaining accuracy.

Contribution

It proposes a new emulator embedded neural network that trains almost instantly by only adjusting last layer weights via linear regression, enabling scalable uncertainty quantification.

Findings

Rapid neural networks train near-instantaneously.

The method maintains prediction accuracy comparable to traditional training.

Demonstrated effectiveness on aerospace design examples.

Abstract

Emulator embedded neural networks, which are a type of physics informed neural network, leverage multi-fidelity data sources for efficient design exploration of aerospace engineering systems. Multiple realizations of the neural network models are trained with different random initializations. The ensemble of model realizations is used to assess epistemic modeling uncertainty caused due to lack of training samples. This uncertainty estimation is crucial information for successful goal-oriented adaptive learning in an aerospace system design exploration. However, the costs of training the ensemble models often become prohibitive and pose a computational challenge, especially when the models are not trained in parallel during adaptive learning. In this work, a new type of emulator embedded neural network is presented using the rapid neural network paradigm. Unlike the conventional neural network training that optimizes the weights and biases of all the network layers by using gradient-based backpropagation, rapid neural network training adjusts only the last layer connection weights by applying a linear regression technique. It is found that the proposed emulator embedded neural network trains near-instantaneously, typically without loss of prediction accuracy. The proposed method is demonstrated on multiple analytical examples, as well as an aerospace flight parameter study of a generic hypersonic vehicle.