A Theory of Nonparametric Covariance Function Estimation for Discretely Observed Data

TL;DR

This paper develops a theoretical framework for nonparametric covariance function estimation from discretely observed noisy data, demonstrating how deep learning estimators can adaptively mitigate the curse of dimensionality in high-dimensional settings.

Contribution

It establishes an oracle inequality for learning-based estimators, derives convergence rates for deep learning methods, and compares their performance with traditional estimators across different function classes.

Findings

Deep learning estimators achieve near-minimax rates for structured covariance functions.

Local linear smoothing outperforms deep learning in one-dimensional smoothness classes.

Structural adaptation helps mitigate the curse of dimensionality in covariance estimation.

Abstract

We study nonparametric covariance function estimation for functional data observed with noise at discrete locations on a $d$-dimensional domain. Estimating the covariance function from discretely observed data is a challenging nonparametric problem, particularly in multidimensional settings, since the covariance function is defined on a product domain and thus suffers from the curse of dimensionality. This motivates the use of adaptive estimators, such as deep learning estimators. However, existing theoretical results are largely limited to estimators with explicit analytic representations, and the properties of general learning-based estimators remain poorly understood. We establish an oracle inequality for a broad class of learning-based estimators that applies to both sparse and dense observation regimes in a unified manner, and derive convergence rates for deep learning estimators over several classes of covariance functions. The resulting rates suggest that structural adaptation can mitigate the curse of dimensionality, similarly to classical nonparametric regression. We further compare the convergence rates of learning-based estimators with several existing procedures. For a one-dimensional smoothness class, deep learning estimators are suboptimal, whereas local linear smoothing estimators achieve a faster rate. For a structured function class, however, deep learning estimators attain the minimax rate up to polylogarithmic factors, whereas local linear smoothing estimators are suboptimal. These results reveal a distinctive adaptivity-variance trade-off in covariance function estimation.