Deep ReLU network approximation of functions on a manifold

TL;DR

This paper investigates how deep ReLU networks can efficiently approximate functions on manifolds, providing theoretical error bounds and convergence rates that depend on the intrinsic manifold dimension rather than the ambient space.

Contribution

It establishes approximation rates for deep ReLU networks on manifolds and derives statistical convergence rates for related estimators, extending understanding of neural network approximation in geometric settings.

Findings

Deep ReLU networks approximate functions on manifolds with error depending on intrinsic dimension.

Approximation error scales as psilon^{-d^*/eta}\

Statistical convergence rates are derived for empirical risk minimizers.

Abstract

Whereas recovery of the manifold from data is a well-studied topic, approximation rates for functions defined on manifolds are less known. In this work, we study a regression problem with inputs on a $d^*$-dimensional manifold that is embedded into a space with potentially much larger ambient dimension. It is shown that sparsely connected deep ReLU networks can approximate a H\"older function with smoothness index $\beta$ up to error $\epsilon$ using of the order of $\epsilon^{-d^*/\beta}\log(1/\epsilon)$ many non-zero network parameters. As an application, we derive statistical convergence rates for the estimator minimizing the empirical risk over all possible choices of bounded network parameters.