Adaptive deep learning for nonlinear time series models

TL;DR

This paper develops a theoretical framework for adaptive deep neural network estimators to accurately model and estimate the mean functions of complex, non-stationary nonlinear time series, achieving near-optimal rates.

Contribution

It introduces a general theory for adaptive DNN-based estimation in nonlinear, non-stationary time series, establishing error bounds and minimax optimality for a broad class of models.

Findings

Sparse-penalized DNN estimators are adaptive and nearly minimax optimal.

Numerical simulations confirm the effectiveness of DNN methods for complex nonlinear AR models.

Theoretical results include generalization error bounds and minimax lower bounds.

Abstract

In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of DNN estimators, non-penalized and sparse-penalized DNN estimators, and establish their generalization error bounds for general non-stationary time series. We then derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models that include nonlinear generalized additive AR, single index, and threshold AR models. Building upon the results, we show that the sparse-penalized DNN estimator is adaptive and attains the minimax optimal rates up to a poly-logarithmic factor for many nonlinear AR models. Through numerical simulations, we demonstrate the usefulness of the DNN methods for estimating nonlinear AR models with intrinsic low-dimensional structures and discontinuous or rough mean functions, which is consistent with our theory.