DPM: A deep learning PDE augmentation method (with application to large-eddy simulation)

TL;DR

This paper introduces a deep learning PDE augmentation framework that embeds neural networks into physical PDEs to improve scientific predictions, especially in turbulence modeling, by leveraging known physics and optimizing the PDE form.

Contribution

The paper presents a novel deep learning PDE model (DPM) that integrates neural networks into PDEs, enabling better out-of-sample predictions and correction of physics errors, with efficient training via adjoint methods.

Findings

DPM outperforms traditional turbulence models in LES.

DPM surpasses a priori trained models by incorporating full PDE physics.

The approach effectively handles large filter sizes where classical models fail.

Abstract

Machine learning for scientific applications faces the challenge of limited data. We propose a framework that leverages a priori known physics to reduce overfitting when training on relatively small datasets. A deep neural network is embedded in a partial differential equation (PDE) that expresses the known physics and learns to describe the corresponding unknown or unrepresented physics from the data. Crafted as such, the neural network can also provide corrections for erroneously represented physics, such as discretization errors associated with the PDE's numerical solution. Once trained, the deep learning PDE model (DPM) can make out-of-sample predictions for new physical parameters, geometries, and boundary conditions. Our approach optimizes over the functional form of the PDE. Estimating the embedded neural network requires optimizing over the entire PDE, which itself is a function of the neural network. Adjoint partial differential equations are used to efficiently calculate the high-dimensional gradient of the objective function with respect to the neural network parameters. A stochastic adjoint method (SAM), similar in spirit to stochastic gradient descent, further accelerates training. The approach is demonstrated and evaluated for turbulence predictions using large-eddy simulation (LES), a filtered version of the Navier--Stokes equation containing unclosed sub-filter-scale terms. The DPM outperforms the widely-used constant-coefficient and dynamic Smagorinsky models, even for filter sizes so large that these established models become qualitatively incorrect. It also significantly outperforms a priori trained models, which do not account for the full PDE. A relaxation of the discrete enforcement of the divergence-free constraint is also considered, instead allowing the DPM to approximately enforce incompressibility physics.