Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks

TL;DR

This paper introduces Lagrangian PINNs (LPINNs), a causality-conforming approach for solving nonlinear convection-diffusion PDEs, which improves training stability and reduces sensitivity to problem complexity compared to traditional Eulerian PINNs.

Contribution

The paper proposes LPINNs, a novel Lagrangian reformulation of PINNs that aligns with the causality of transport phenomena, enhancing training robustness and addressing failure modes of standard PINNs.

Findings

LPINNs exhibit less sensitivity to problem complexity.

LPINNs conform to the causality of convection processes.

Training landscapes of LPINNs are more stable than traditional PINNs.

Abstract

Physics-informed neural networks (PINNs) leverage neural-networks to find the solutions of partial differential equation (PDE)-constrained optimization problems with initial conditions and boundary conditions as soft constraints. These soft constraints are often considered to be the sources of the complexity in the training phase of PINNs. Here, we demonstrate that the challenge of training (i) persists even when the boundary conditions are strictly enforced, and (ii) is closely related to the Kolmogorov n-width associated with problems demonstrating transport, convection, traveling waves, or moving fronts. Given this realization, we describe the mechanism underlying the training schemes such as those used in eXtended PINNs (XPINN), curriculum regularization, and sequence-to-sequence learning. For an important category of PDEs, i.e., governed by non-linear convection-diffusion equation, we propose reformulating PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed solution. A parallel architecture with two branches is proposed. One branch solves for the state variables on the characteristics, and the second branch solves for the low-dimensional characteristics curves. The proposed architecture conforms to the causality innate to the convection, and leverages the direction of travel of the information in the domain. Finally, we demonstrate that the loss landscapes of LPINNs are less sensitive to the so-called "complexity" of the problems, compared to those in the traditional PINNs in the Eulerian framework.