Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces

TL;DR

This paper demonstrates that deep ReLU CNNs can achieve higher order approximation rates for Korobov functions, effectively mitigating the curse of dimensionality through sparse grid basis representations.

Contribution

It improves classical approximation rates for Korobov functions using CNNs, showing higher order expressivity with minimal curse of dimensionality effects.

Findings

Achieves (m+1)-th order approximation rate in CNNs for functions with (m+1)-th order derivatives.

Uses sparse grid basis functions to represent high-order derivatives within CNNs.

Suggests higher order CNN expressivity is not severely impacted by high dimensionality.

Abstract

This paper investigates the $L_p$ approximation error for higher order Korobov functions using deep convolutional neural networks (CNNs) with ReLU activation. For target functions having a mixed derivative of order m+1 in each direction, we improve classical approximation rate of second order to (m+1)-th order (modulo a logarithmic factor) in terms of the depth of CNNs. The key ingredient in our analysis is approximate representation of high-order sparse grid basis functions by CNNs. The results suggest that higher order expressivity of CNNs does not severely suffer from the curse of dimensionality.