Do physics-informed neural networks (PINNs) need to be deep? Shallow PINNs using the Levenberg-Marquardt algorithm

TL;DR

This paper demonstrates that shallow physics-informed neural networks optimized with the Levenberg-Marquardt algorithm can effectively solve complex PDEs, outperforming traditional methods in speed and accuracy.

Contribution

It introduces a novel approach using shallow PINNs with LM optimization, showing significant improvements over existing methods for solving nonlinear PDEs.

Findings

Shallow PINNs with LM outperform BFGS in convergence speed.

The approach achieves high accuracy with only two hidden layers.

Numerical results on benchmark PDEs validate the method's efficiency.

Abstract

This work investigates the use of shallow physics-informed neural networks (PINNs) for solving forward and inverse problems of nonlinear partial differential equations (PDEs). By reformulating PINNs as nonlinear systems, the Levenberg-Marquardt (LM) algorithm is employed to efficiently optimize the network parameters. Analytical expressions for the neural network derivatives with respect to the input variables are derived, enabling accurate and efficient computation of the Jacobian matrix required by LM. The proposed approach is tested on several benchmark problems, including the Burgers, Schr\"odinger, Allen-Cahn, and three-dimensional Bratu equations. Numerical results demonstrate that LM significantly outperforms BFGS in terms of convergence speed, accuracy, and final loss values, even when using shallow network architectures with only two hidden layers. These findings indicate that, for a wide class of PDEs, shallow PINNs combined with efficient second-order optimization methods can provide accurate and computationally efficient solutions for both forward and inverse problems.