On the convergence of PINNs

TL;DR

This paper investigates the theoretical properties of physics-informed neural networks (PINNs), demonstrating that regularization techniques can improve their convergence and physical consistency in solving PDEs.

Contribution

It provides a rigorous analysis of PINNs' convergence and introduces regularization methods to enhance their physical and statistical accuracy.

Findings

Regularization improves PINNs' risk consistency.

Sobolev-type regularization reconstructs physically consistent solutions.

Classical training can suffer from systematic overfitting.

Abstract

Physics-informed neural networks (PINNs) are a promising approach that combines the power of neural networks with the interpretability of physical modeling. PINNs have shown good practical performance in solving partial differential equations (PDEs) and in hybrid modeling scenarios, where physical models enhance data-driven approaches. However, it is essential to establish their theoretical properties in order to fully understand their capabilities and limitations. In this study, we highlight that classical training of PINNs can suffer from systematic overfitting. This problem can be addressed by adding a ridge regularization to the empirical risk, which ensures that the resulting estimator is risk-consistent for both linear and nonlinear PDE systems. However, the strong convergence of PINNs to a solution satisfying the physical constraints requires a more involved analysis using tools from functional analysis and calculus of variations. In particular, for linear PDE systems, an implementable Sobolev-type regularization allows to reconstruct a solution that not only achieves statistical accuracy but also maintains consistency with the underlying physics.