Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications

TL;DR

This paper introduces a novel Bayesian neural network training method called anchored ensembling, which effectively incorporates prior knowledge in mechanics surrogate modeling, improving uncertainty quantification and predictive accuracy.

Contribution

The work presents a new anchored ensembling approach for Bayesian neural networks that leverages functional priors and weight correlations, enhancing prior integration and uncertainty estimation.

Findings

Improved uncertainty quantification in surrogate models.

Effective transfer of prior knowledge from low-fidelity models.

Enhanced accuracy in both interpolation and extrapolation tasks.

Abstract

In recent years, neural networks (NNs) have become increasingly popular for surrogate modeling tasks in mechanics and materials modeling applications. While traditional NNs are deterministic functions that rely solely on data to learn the input--output mapping, casting NN training within a Bayesian framework allows to quantify uncertainties, in particular epistemic uncertainties that arise from lack of training data, and to integrate a priori knowledge via the Bayesian prior. However, the high dimensionality and non-physicality of the NN parameter space, and the complex relationship between parameters (NN weights) and predicted outputs, renders both prior design and posterior inference challenging. In this work we present a novel BNN training scheme based on anchored ensembling that can integrate a priori information available in the function space, from e.g. low-fidelity models. The anchoring scheme makes use of low-rank correlations between NN parameters, learnt from pre-training to realizations of the functional prior. We also perform a study to demonstrate how correlations between NN weights, which are often neglected in existing BNN implementations, is critical to appropriately transfer knowledge between the function-space and parameter-space priors. Performance of our novel BNN algorithm is first studied on a small 1D example to illustrate the algorithm's behavior in both interpolation and extrapolation settings. Then, a thorough assessment is performed on a multi--input--output materials surrogate modeling example, where we demonstrate the algorithm's capabilities both in terms of accuracy and quality of the uncertainty estimation, for both in-distribution and out-of-distribution data.