Scaling Laws and Pathologies of Single-Layer PINNs: Network Width and PDE Nonlinearity

TL;DR

This paper investigates how single-layer PINNs perform on nonlinear PDEs, revealing that network width and nonlinearity cause optimization failures that limit accuracy, with complex scaling laws governing these effects.

Contribution

It uncovers dual optimization failures in single-layer PINNs and demonstrates that their scaling behavior is governed by complex, non-separable relationships influenced by spectral bias.

Findings

Network width does not always improve solution error due to optimization failures.

Nonlinearity exacerbates the difficulty in learning high-frequency solution components.

Optimization, not capacity, is the main bottleneck in training PINNs for nonlinear PDEs.

Abstract

We establish empirical scaling laws for Single-Layer Physics-Informed Neural Networks on canonical nonlinear PDEs. We identify a dual optimization failure: (i) a baseline pathology, where the solution error fails to decrease with network width, even at fixed nonlinearity, falling short of theoretical approximation bounds, and (ii) a compounding pathology, where this failure is exacerbated by nonlinearity. We provide quantitative evidence that a simple separable power law is insufficient, and that the scaling behavior is governed by a more complex, non-separable relationship. This failure is consistent with the concept of spectral bias, where networks struggle to learn the high-frequency solution components that intensify with nonlinearity. We show that optimization, not approximation capacity, is the primary bottleneck, and propose a methodology to empirically measure these complex scaling effects.