Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions

TL;DR

This paper establishes nearly optimal super-approximation error bounds for ReLU neural networks approximating Korobov functions, showing that neural network expressivity can overcome the curse of dimensionality in certain norms.

Contribution

It derives new super-approximation error bounds for ReLU neural networks on Korobov functions, improving classical bounds and highlighting their ability to bypass the curse of dimensionality.

Findings

Achieves nearly optimal super-approximation rates in $L_p$ and $W^1_p$ norms.

Demonstrates neural networks' expressivity is largely unaffected by the curse of dimensionality.

Uses sparse grid finite elements and bit extraction techniques for analysis.

Abstract

This paper examines the $L_p$ and $W^1_p$ norm approximation errors of ReLU neural networks for Korobov functions. In terms of network width and depth, we derive nearly optimal super-approximation error bounds of order $2m$ in the $L_p$ norm and order $2m-2$ in the $W^1_p$ norm, for target functions with $L_p$ mixed derivative of order $m$ in each direction. The analysis leverages sparse grid finite elements and the bit extraction technique. Our results improve upon classical lowest order $L_\infty$ and $H^1$ norm error bounds and demonstrate that the expressivity of neural networks is largely unaffected by the curse of dimensionality.