A general framework for deep learning

TL;DR

This paper introduces a comprehensive framework for deep learning applicable to various data dependencies, proposing two estimators with proven minimax optimality bounds for nonparametric regression and classification tasks.

Contribution

It develops a unified approach for deep learning under diverse data dependence structures, proposing two estimators with theoretical risk bounds and demonstrating their optimality.

Findings

Both estimators achieve minimax optimal risk bounds in classical settings.

The framework applies to independent, mixing, and strongly mixing data.

Bounds are established for H"older and composition H"older functions.

Abstract

This paper develops a general approach for deep learning for a setting that includes nonparametric regression and classification. We perform a framework from data that fulfills a generalized Bernstein-type inequality, including independent, $\phi$-mixing, strongly mixing and $\mathcal{C}$-mixing observations. Two estimators are proposed: a non-penalized deep neural network estimator (NPDNN) and a sparse-penalized deep neural network estimator (SPDNN). For each of these estimators, bounds of the expected excess risk on the class of H\"older smooth functions and composition H\"older functions are established. Applications to independent data, as well as to $\phi$-mixing, strongly mixing, $\mathcal{C}$-mixing processes are considered. For each of these examples, the upper bounds of the expected excess risk of the proposed NPDNN and SPDNN predictors are derived. It is shown that both the NPDNN and SPDNN estimators are minimax optimal (up to a logarithmic factor) in many classical settings.