A theory of nonparametric regression in the presence of complex nuisance components

TL;DR

This paper develops a geometric approach to nonparametric regression with complex nuisance components, providing risk bounds for estimating a target function even when nuisance parts are high-dimensional or less smooth.

Contribution

It introduces a Hilbert space geometric framework that yields new risk bounds for estimating target functions amid complex nuisance components, applicable to high-dimensional and increasing component scenarios.

Findings

Risk bounds depend on geometric quantities like angles and norms.

Estimates achieve sharp upper bounds similar to simpler models.

Applicable to models with many or less smooth nuisance components.

Abstract

In this paper, we consider the nonparametric random regression model $Y=f_1(X_1)+f_2(X_2)+\epsilon$ and address the problem of estimating the function $f_1$. The term $f_2(X_2)$ is regarded as a nuisance term which can be considerably more complex than $f_1(X_1)$. Under minimal assumptions, we prove several nonasymptotic $L^2(\mathbb{P}^X)$-risk bounds for our estimators of $f_1$. Our approach is geometric and based on considerations in Hilbert spaces. It shows that the performance of our estimators is closely related to geometric quantities, such as minimal angles and Hilbert-Schmidt norms. Our results establish new conditions under which the estimators of $f_1$ have up to first order the same sharp upper bound as the corresponding estimators of $f_1$ in the model $Y=f_1(X_1)+\epsilon$. As an example we apply the results to an additive model in which the number of components is very large or in which the nuisance components are considerably less smooth than $f_1$. In particular, the results apply to an asymptotic scenario in which the number of components is allowed to increase with the sample size.