# Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery

## TL;DR

This paper demonstrates that deep ReLU networks can effectively perform nonparametric regression on data supported on low-dimensional manifolds, achieving fast convergence rates that depend on the intrinsic dimension rather than the ambient space.

## Contribution

The paper introduces a deep ReLU network architecture for nonparametric regression on manifolds and proves its convergence rate depends on the intrinsic dimension, showing adaptivity to geometric structures.

## Key findings

- Convergence rate of $n^{-rac{2(s+eta)}{2(s+eta)+d}}\
- Deep ReLU networks adapt to low-dimensional manifold structures in high-dimensional data.
- Theoretical analysis supports the effectiveness of deep networks for geometric data approximation.

## Abstract

Real world data often exhibit low-dimensional geometric structures, and can be viewed as samples near a low-dimensional manifold. This paper studies nonparametric regression of H\"{o}lder functions on low-dimensional manifolds using deep ReLU networks. Suppose $n$ training data are sampled from a H\"{o}lder function in $\mathcal{H}^{s,\alpha}$ supported on a $d$-dimensional Riemannian manifold isometrically embedded in $\mathbb{R}^D$, with sub-gaussian noise. A deep ReLU network architecture is designed to estimate the underlying function from the training data. The mean squared error of the empirical estimator is proved to converge in the order of $n^{-\frac{2(s+\alpha)}{2(s+\alpha) + d}}\log^3 n$. This result shows that deep ReLU networks give rise to a fast convergence rate depending on the data intrinsic dimension $d$, which is usually much smaller than the ambient dimension $D$. It therefore demonstrates the adaptivity of deep ReLU networks to low-dimensional geometric structures of data, and partially explains the power of deep ReLU networks in tackling high-dimensional data with low-dimensional geometric structures.