Spectral Barron space for deep neural network approximation

TL;DR

This paper establishes a dimension-independent relationship between spectral Barron and Besov spaces and demonstrates how neural network approximation errors scale with network size and depth for functions in the spectral Barron space.

Contribution

It introduces a sharp embedding between spectral Barron and Besov spaces and provides a dimension-free convergence rate for neural network approximation errors.

Findings

Embedding constants are independent of input dimension.

Approximation error scales as N^{-sL} with network size and depth.

Provides theoretical bounds for neural network approximation in spectral Barron space.

Abstract

We prove the sharp embedding between the spectral Barron space and the Besov space with embedding constants independent of the input dimension. Given the spectral Barron space as the target function space, we prove a dimension-free convergence result that if the neural network contains $L$ hidden layers with $N$ units per layer, then the upper and lower bounds of the $L^2$-approximation error are $\mathcal{O}(N^{-sL})$ with $0 < sL\le 1/2$, where $s\ge 0$ is the smoothness index of the spectral Barron space.