Physics Constrained Learning for Data-driven Inverse Modeling from Sparse Observations

TL;DR

This paper introduces a physics-constrained learning method that integrates PDE solvers into neural network training, leading to faster convergence and improved stability in inverse modeling with sparse data.

Contribution

The authors develop a novel algorithm for differentiating both explicit and implicit PDE solvers within neural network training, enabling direct enforcement of physical constraints.

Findings

Faster convergence compared to penalty methods.

Enhanced stability in stiff problems.

Potential for broad application in inverse modeling.

Abstract

Deep neural networks (DNN) have been used to model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that measures the discrepancy between predictions and observations in some chosen norm. This loss function often includes the PDE constraints as a penalty term when only sparse observations are available. As a result, the PDE is only satisfied approximately by the solution. However, the penalty term typically slows down the convergence of the optimizer for stiff problems. We present a new approach that trains the embedded DNNs while numerically satisfying the PDE constraints. We develop an algorithm that enables differentiating both explicit and implicit numerical solvers in reverse-mode automatic differentiation. This allows the gradients of the DNNs and the PDE solvers to be computed in a unified framework. We demonstrate that our approach enjoys faster convergence and better stability in relatively stiff problems compared to the penalty method. Our approach allows for the potential to solve and accelerate a wide range of data-driven inverse modeling, where the physical constraints are described by PDEs and need to be satisfied accurately.