Convergence Guarantees for Gradient-Based Training of Neural PDE Solvers: From Linear to Nonlinear PDEs

TL;DR

This paper develops a comprehensive convergence theory for gradient-based neural PDE solvers, covering linear and nonlinear cases, and reveals implicit regularization effects through theoretical analysis and numerical experiments.

Contribution

It extends the neural tangent kernel framework to linear PDEs and proves convergence for nonlinear PDEs without strong over-parameterization, unifying theory for neural PDE training.

Findings

Global convergence guarantees for linear PDEs via NTK extension.

Convergence to critical points for nonlinear PDEs under the ojasiewicz inequality.

Implicit regularization prevents parameter divergence in neural PDE training.

Abstract

We present a unified convergence theory for gradient-based training of neural network methods for partial differential equations (PDEs), covering both physics-informed neural networks (PINNs) and the Deep Ritz method. For linear PDEs, we extend the neural tangent kernel (NTK) framework for PINNs to establish global convergence guarantees for a broad class of linear operators. For nonlinear PDEs, we prove convergence to critical points via the \L{}ojasiewicz inequality under the random feature model, eliminating the need for strong over-parameterization and encompassing both gradient flow and implicit gradient descent dynamics. Our results further reveal that the random feature model exhibits an implicit regularization effect, preventing parameter divergence to infinity. Theoretical findings are corroborated by numerical experiments, providing new insights into the training dynamics and robustness of neural network PDE solvers.