Estimation of a regression function on a manifold by fully connected deep neural networks

TL;DR

This paper analyzes the convergence rates of deep neural network estimators for regression functions when predictors lie on a manifold, showing they depend on the manifold's dimension rather than the ambient space.

Contribution

It demonstrates that fully connected deep neural networks with ReLU can effectively estimate regression functions on manifolds, achieving convergence rates based on manifold dimension.

Findings

Convergence rate depends on manifold dimension, not ambient space.

Deep neural networks achieve optimal rates for smooth functions on manifolds.

Estimates are effective for predictor distributions concentrated on manifolds.

Abstract

Estimation of a regression function from independent and identically distributed data is considered. The $L_2$ error with integration with respect to the distribution of the predictor variable is used as the error criterion. The rate of convergence of least squares estimates based on fully connected spaces of deep neural networks with ReLU activation function is analyzed for smooth regression functions. It is shown that in case that the distribution of the predictor variable is concentrated on a manifold, these estimates achieve a rate of convergence which depends on the dimension of the manifold and not on the number of components of the predictor variable.