Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory

TL;DR

This paper analyzes how gradient descent can find global solutions in training two-layer neural networks for second-order linear PDEs and studies their generalization properties under Barron space assumptions.

Contribution

It provides theoretical guarantees for global minimization via gradient descent and analyzes generalization error without over-parametrization assumptions.

Findings

Gradient descent can identify global minimizers for over-parameterized networks solving PDEs.

Generalization error bounds are established for networks with Barron space regularization.

The analysis applies to second-order linear PDEs with neural network parametrizations.

Abstract

The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions. A global minimizer corresponds to a neural network that solves the given PDE. In this paper, we show that the gradient descent method can identify a global minimizer of the least-squares optimization for solving second-order linear PDEs with two-layer neural networks under the assumption of over-parametrization. We also analyze the generalization error of the least-squares optimization for second-order linear PDEs and two-layer neural networks, when the right-hand-side function of the PDE is in a Barron-type space and the least-squares optimization is regularized with a Barron-type norm, without the over-parametrization assumption.