Bayesian Entropy Neural Networks for Physics-Aware Prediction

TL;DR

This paper introduces Bayesian Entropy Neural Networks (BENN), a novel framework that integrates constraints into Bayesian Neural Networks using MaxEnt principles, improving robustness and uncertainty quantification in physics-aware predictions.

Contribution

The paper presents BENN, a new method combining MaxEnt and multipliers to impose constraints on BNNs, enabling better physics-aware modeling with limited data and partial information.

Findings

BENN effectively constrains predictions, derivatives, and variances.

Experimental results show improved accuracy over traditional BNNs.

BENN performs competitively with state-of-the-art constrained deep learning methods.

Abstract

This paper addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring flexible model behavior to incorporate non-data sample information. We introduce Bayesian Entropy Neural Networks (BENN), a framework grounded in Maximum Entropy (MaxEnt) principles, designed to impose constraints on Bayesian Neural Network (BNN) predictions. BENN is capable of constraining not only the predicted values but also their derivatives and variances, ensuring a more robust and reliable model output. To achieve simultaneous uncertainty quantification and constraint satisfaction, we employ the method of multipliers approach. This allows for the concurrent estimation of neural network parameters and the Lagrangian multipliers associated with the constraints. Our experiments, spanning diverse applications such as beam deflection modeling and microstructure generation, demonstrate the effectiveness of BENN. The results highlight significant improvements over traditional BNNs and showcase competitive performance relative to contemporary constrained deep learning methods.