Noisy PDE Training Requires Bigger PINNs

TL;DR

This paper establishes theoretical lower bounds on the size of Physics-Informed Neural Networks (PINNs) needed to effectively learn PDE solutions from noisy data, supported by empirical experiments on various PDEs.

Contribution

It provides the first quantitative analysis linking network size, data noise, and empirical risk in PINNs, revealing that larger models are necessary for noisy data.

Findings

PINNs require a minimum size related to data noise and sample count to achieve low empirical risk.

Increasing noisy data alone does not reduce empirical risk without larger model capacity.

Empirical results confirm the theoretical bounds across multiple PDEs.

Abstract

Physics-Informed Neural Networks (PINNs) are increasingly used to approximate solutions of partial differential equations (PDEs), particularly in high dimensions. In real-world settings, data are often noisy, making it crucial to understand when a predictor can still achieve low empirical risk. Yet, little is known about the conditions under which a PINN can do so effectively. We analyse PINNs applied to the Hamilton--Jacobi--Bellman (HJB) PDE and establish a lower bound on the network size required for the supervised PINN empirical risk to fall below the variance of noisy supervision labels. Specifically, if a predictor achieves empirical risk $O(\eta)$ below $\sigma^2$ (the variance of the supervision data), then necessarily $d_N\log d_N\gtrsim N_s \eta^2$, where $N_s$ is the number of samples and $d_N$ the number of trainable parameters. A similar constraint holds in the fully unsupervised PINN setting when boundary labels are noisy. Thus, simply increasing the number of noisy supervision labels does not offer a ``free lunch'' in reducing empirical risk. We also give empirical studies on the HJB PDE, the Poisson PDE and the the Navier-Stokes PDE set to produce the Taylor-Green solutions. In these experiments we demonstrate that PINNs indeed need to be beyond a threshold model size for them to train to errors below $\sigma^2$. These results provide a quantitative foundation for understanding parameter requirements when training PINNs in the presence of noisy data.