Penalized deep neural networks estimator with general loss functions under weak dependence

TL;DR

This paper develops a framework for sparse-penalized deep neural networks to learn weakly dependent processes across various tasks, providing non-asymptotic bounds and convergence rates, with applications to time series forecasting.

Contribution

It introduces a general approach for deep neural network estimation under weak dependence, including new non-asymptotic bounds and oracle inequalities for broad classes of loss functions.

Findings

Non-asymptotic generalization bounds for weakly dependent data.

Convergence rate close to O(n^{-1/3}) for smooth target functions.

Successful application to particulate matter forecasting in Vitória.

Abstract

This paper carries out sparse-penalized deep neural networks predictors for learning weakly dependent processes, with a broad class of loss functions. We deal with a general framework that includes, regression estimation, classification, times series prediction, $\cdots$ The $\psi$-weak dependence structure is considered, and for the specific case of bounded observations, $\theta_\infty$-coefficients are also used. In this case of $\theta_\infty$-weakly dependent, a non asymptotic generalization bound within the class of deep neural networks predictors is provided. For learning both $\psi$ and $\theta_\infty$-weakly dependent processes, oracle inequalities for the excess risk of the sparse-penalized deep neural networks estimators are established. When the target function is sufficiently smooth, the convergence rate of these excess risk is close to $\mathcal{O}(n^{-1/3})$. Some simulation results are provided, and application to the forecast of the particulate matter in the Vit\'{o}ria metropolitan area is also considered.