A Priori Estimation of the Approximation, Optimization and Generalization Errors of Random Neural Networks for Solving Partial Differential Equations

TL;DR

This paper provides a theoretical framework for estimating the approximation, optimization, and generalization errors of random neural networks in solving PDEs, focusing on Barron functions and Sobolev norms.

Contribution

It introduces a priori bounds for errors in random neural networks applied to PDEs, clarifying the role of prior distributions and extending understanding of their theoretical performance.

Findings

Derived bounds for approximation, optimization, and generalization errors.

Focused on Barron type functions and Sobolev norm approximations.

Provided insights into prior distribution selection for random neural networks.

Abstract

In recent years, neural networks have achieved remarkable progress in various fields and have also drawn much attention in applying them on scientific problems. A line of methods involving neural networks for solving partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM), has emerged. Although these methods outperform classical numerical methods in certain cases, the optimization problems involving neural networks are typically non-convex and non-smooth, which can result in unsatisfactory solutions for PDEs. In contrast to deterministic neural networks, the hidden weights of random neural networks are sampled from some prior distribution and only the output weights participate in training. This makes training much simpler, but it remains unclear how to select the prior distribution. In this paper, we focus on Barron type functions and approximate them under Sobolev norms by random neural networks with clear prior distribution. In addition to the approximation error, we also derive bounds for the optimization and generalization errors of random neural networks for solving PDEs when the solutions are Barron type functions.