Deep regression learning from dependent observations with minimum error entropy principle

TL;DR

This paper introduces deep neural network estimators based on the minimum error entropy principle for nonparametric regression with dependent, strongly mixing data, achieving near-optimal convergence rates.

Contribution

It develops and analyzes MEE-based deep neural network estimators, providing theoretical guarantees and establishing their minimax optimal convergence rates under dependent data.

Findings

Both estimators achieve minimax optimal rates for Gaussian errors.

Upper bounds on expected excess risk are established for H"older function classes.

The methods work effectively with strongly mixing observations.

Abstract

This paper considers nonparametric regression from strongly mixing observations. The proposed approach is based on deep neural networks with minimum error entropy (MEE) principle. We study two estimators: the non-penalized deep neural network (NPDNN) and the sparse-penalized deep neural network (SPDNN) predictors. Upper bounds of the expected excess risk are established for both estimators over the classes of H\"older and composition H\"older functions. For the models with Gaussian error, the rates of the upper bound obtained match (up to a logarithmic factor) with the lower bounds established in \cite{schmidt2020nonparametric}, showing that both the MEE-based NPDNN and SPDNN estimators from strongly mixing data can achieve the minimax optimal convergence rate.