Optimization and generalization analysis for two-layer physics-informed neural networks without over-parametrization

TL;DR

This paper analyzes the optimization and generalization of two-layer physics-informed neural networks trained with SGD, demonstrating that under certain conditions, the loss can be reduced below a specified threshold without over-parameterization.

Contribution

It provides a new analysis of SGD training for two-layer PINNs under non-over-parameterized regimes, avoiding the computational costs of over-parameterization.

Findings

Training loss decreases below O(ε) with sufficient network width.

Analysis avoids reliance on over-parameterization assumptions.

Results applicable to practical PINN training scenarios.

Abstract

This work focuses on the behavior of stochastic gradient descent (SGD) in solving least-squares regression with physics-informed neural networks (PINNs). Past work on this topic has been based on the over-parameterization regime, whose convergence may require the network width to increase vastly with the number of training samples. So, the theory derived from over-parameterization may incur prohibitive computational costs and is far from practical experiments. We perform new optimization and generalization analysis for SGD in training two-layer PINNs, making certain assumptions about the target function to avoid over-parameterization. Given $\epsilon>0$, we show that if the network width exceeds a threshold that depends only on $\epsilon$ and the problem, then the training loss and expected loss will decrease below $O(\epsilon)$.