Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks

TL;DR

This paper provides a non-asymptotic convergence analysis for projected gradient descent applied to physics-informed neural networks solving the Poisson equation, offering explicit error bounds and insights into optimization and generalization errors.

Contribution

It introduces a novel non-asymptotic analysis framework for physics-informed neural networks, combining optimization and generalization error bounds under suitable assumptions.

Findings

Optimization error bounded by (1/T + 1/m + _{ ext{approx}})

Bounding of Rademacher complexities for neural networks and their Laplacian

Overall error estimate derived from optimization and regularity theory

Abstract

In this work, we provide a non-asymptotic convergence analysis of projected gradient descent for physics-informed neural networks for the Poisson equation. Under suitable assumptions, we show that the optimization error can be bounded by $\mathcal{O}(1/\sqrt{T} + 1/\sqrt{m} + \epsilon_{\text{approx}})$, where $T$ is the number of algorithm time steps, $m$ is the width of the neural network and $\epsilon_{\text{approx}}$ is an approximation error. The proof of our optimization result relies on bounding the linearization error and using this result together with a Lyapunov drift analysis. Additionally, we quantify the generalization error by bounding the Rademacher complexities of the neural network and its Laplacian. Combining both the optimization and generalization results, we obtain an overall error estimate based on an existing error estimate from regularity theory.