# Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks

## TL;DR

This paper proves that stochastic gradient descent converges linearly when training over-parameterized two-layer physics-informed neural networks for PDEs, providing theoretical guarantees for stochastic training methods.

## Contribution

It extends previous convergence results from gradient descent to stochastic gradient descent for PINNs, handling dynamic randomness in training.

## Key findings

- Establishes linear convergence of SGD for over-parameterized PINNs
- Handles general activation functions in the analysis
- Provides high probability convergence guarantees

## Abstract

Physics informed neural networks (PINNs) represent a very popular class of neural solvers for partial differential equations. In practice, one often employs stochastic gradient descent type algorithms to train the neural network. Therefore, the convergence guarantee of stochastic gradient descent is of fundamental importance. In this work, we establish the linear convergence of stochastic gradient descent / flow in training over-parameterized two layer PINNs for a general class of activation functions in the sense of high probability. These results extend the existing result [18] in which gradient descent was analyzed. The challenge of the analysis lies in handling the dynamic randomness introduced by stochastic optimization methods. The key of the analysis lies in ensuring the positive definiteness of suitable Gram matrices during the training. The analysis sheds insight into the dynamics of the optimization process, and provides guarantees on the neural networks trained by stochastic algorithms.