Nonparametric Estimation via Partial Derivatives

TL;DR

This paper introduces a novel nonparametric estimation method using partial derivatives that achieves near-parametric convergence rates, overcoming traditional limitations in high-dimensional settings.

Contribution

The paper develops a new approach based on partial derivatives within the SS-ANOVA framework, enabling faster convergence rates and immunity to the curse of interaction in high-dimensional nonparametric models.

Findings

Gradient information improves convergence rates in high-dimensional models.

Optimal rates for models with full interaction are unaffected by the number of covariates.

Additive models achieve root-n parametric rates with gradient information.

Abstract

Traditional nonparametric estimation methods often lead to a slow convergence rate in large dimensions and require unrealistically enormous sizes of datasets for reliable conclusions. We develop an approach based on partial derivatives, either observed or estimated, to effectively estimate the function at near-parametric convergence rates. The novel approach and computational algorithm could lead to methods useful to practitioners in many areas of science and engineering. Our theoretical results reveal a behavior universal to this class of nonparametric estimation problems. We explore a general setting involving tensor product spaces and build upon the smoothing spline analysis of variance (SS-ANOVA) framework. For $d$-dimensional models under full interaction, the optimal rates with gradient information on $p$ covariates are identical to those for the $(d-p)$-interaction models without gradients and, therefore, the models are immune to the "curse of interaction." For additive models, the optimal rates using gradient information are root-$n$, thus achieving the "parametric rate." We demonstrate aspects of the theoretical results through synthetic and real data applications.