Statistical Learning Analysis of Physics-Informed Neural Networks

TL;DR

This paper analyzes physics-informed neural networks (PINNs) from a statistical learning perspective, revealing their nature as a singular learning problem and exploring implications for uncertainty quantification and extrapolation.

Contribution

It reformulates PINNs training as a statistical learning problem and applies singular learning theory to analyze parameter estimation and uncertainty.

Findings

PINNs can be viewed as fitting residual distributions to true data.

Physics-informed learning acts as an infinite source of indirect data.

PINNs are characterized as a singular learning problem.

Abstract

We study the training and performance of physics-informed learning for initial and boundary value problems (IBVP) with physics-informed neural networks (PINNs) from a statistical learning perspective. Specifically, we restrict ourselves to parameterizations with hard initial and boundary condition constraints and reformulate the problem of estimating PINN parameters as a statistical learning problem. From this perspective, the physics penalty on the IBVP residuals can be better understood not as a regularizing term bus as an infinite source of indirect data, and the learning process as fitting the PINN distribution of residuals $p(y \mid x, t, w) q(x, t) $ to the true data-generating distribution $\delta(0) q(x, t)$ by minimizing the Kullback-Leibler divergence between the true and PINN distributions. Furthermore, this analysis show that physics-informed learning with PINNs is a singular learning problem, and we employ singular learning theory tools, namely the so-called Local Learning Coefficient (Lau et al., 2025) to analyze the estimates of PINN parameters obtained via stochastic optimization for a heat equation IBVP. Finally, we discuss implications of this analysis on the quantification of predictive uncertainty of PINNs and the extrapolation capacity of PINNs.