Optimal Convergence Rates for Neural Operators

TL;DR

This paper establishes optimal convergence rates for neural operators within the neural tangent kernel framework, providing theoretical bounds and practical insights for PDE solution approximation.

Contribution

It introduces the NTK regime for neural operators, deriving minimax optimal convergence rates and bounds on network complexity for generalization.

Findings

Fast convergence rates for neural operators using early-stopped GD

Bounds on hidden neurons and sample size for generalization

Neural operators can be approximated by RKHS operators

Abstract

We introduce the neural tangent kernel (NTK) regime for two-layer neural operators and analyze their generalization properties. For early-stopped gradient descent (GD), we derive fast convergence rates that are known to be minimax optimal within the framework of non-parametric regression in reproducing kernel Hilbert spaces (RKHS). We provide bounds on the number of hidden neurons and the number of second-stage samples necessary for generalization. To justify our NTK regime, we additionally show that any operator approximable by a neural operator can also be approximated by an operator from the RKHS. A key application of neural operators is learning surrogate maps for the solution operators of partial differential equations (PDEs). We consider the standard Poisson equation to illustrate our theoretical findings with simulations.