Deep Regression for Repeated Measurements

TL;DR

This paper develops a theoretical framework for deep neural network estimators in nonparametric mean regression with repeated measurements, addressing clustered dependence and phase transition phenomena, with applications to low-dimensional structures.

Contribution

It introduces a novel theoretical approach for DNN estimators in clustered data, revealing phase transition in convergence rates and demonstrating adaptivity to low-dimensional structures.

Findings

Establishes a new framework for DNN regression with repeated measurements.

Identifies phase transition in convergence rates related to data dependence.

Shows DNN estimators adapt to low intrinsic dimensions, overcoming curse of dimensionality.

Abstract

Nonparametric mean function regression with repeated measurements serves as a cornerstone for many statistical branches, such as longitudinal/panel/functional data analysis. In this work, we investigate this problem using fully connected deep neural network (DNN) estimators with flexible shapes. A novel theoretical framework allowing arbitrary sampling frequency is established by adopting empirical process techniques to tackle clustered dependence. We then consider the DNN estimators for H\"older target function and illustrate a key phenomenon, the phase transition in the convergence rate, inherent to repeated measurements and its connection to the curse of dimensionality. Furthermore, we study several examples with low intrinsic dimensions, including the hierarchical composition model, low-dimensional support set and anisotropic H\"older smoothness. We also obtain new approximation results and matching lower bounds to demonstrate the adaptivity of the DNN estimators for circumventing the curse of dimensionality. Simulations and real data examples are provided to support our theoretical findings and practical implications.