A Lagrangian Dual-based Theory-guided Deep Neural Network

TL;DR

This paper introduces a Lagrangian dual-based approach to enhance theory-guided neural networks by better balancing data and domain knowledge, leading to improved accuracy and efficiency in physical modeling tasks.

Contribution

It proposes a novel Lagrangian dual formulation for TgNNs that optimizes the tradeoff between data and constraints, improving prediction accuracy and computational efficiency.

Findings

Outperforms original TgNN in subsurface flow prediction

Reduces computational time compared to baseline models

Achieves higher accuracy with fewer training resources

Abstract

The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of TgNN. We convert the original loss function into a constrained form with fewer items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable tradeoff between observation data and corresponding constraints, in order to improve prediction accuracy, and conserve time and computational resources adjusted by an ad-hoc procedure. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.