Convergence of the generalization error for deep gradient flow methods for PDEs

TL;DR

This paper establishes a mathematical foundation for deep gradient flow methods in solving high-dimensional PDEs, showing that their generalization error diminishes as neural network size and training duration increase.

Contribution

It provides a rigorous analysis of the convergence of generalization error for DGFMs, including approximation capabilities and the behavior of the training process in the wide network limit.

Findings

Generalization error tends to zero with increasing neurons and training time.

Neural networks can approximate solutions of PDEs under verifiable assumptions.

Gradient flow analysis explains the training dynamics in the infinite-width limit.

Abstract

The aim of this article is to provide a firm mathematical foundation for the application of deep gradient flow methods (DGFMs) for the solution of (high-dimensional) partial differential equations (PDEs). We decompose the generalization error of DGFMs into an approximation and a training error. We first show that the solution of PDEs that satisfy reasonable and verifiable assumptions can be approximated by neural networks, thus the approximation error tends to zero as the number of neurons tends to infinity. Then, we derive the gradient flow that the training process follows in the ``wide network limit'' and analyze the limit of this flow as the training time tends to infinity. These results combined show that the generalization error of DGFMs tends to zero as the number of neurons and the training time tend to infinity.