Nonparametric regression with the scale depending on auxiliary variable

TL;DR

This paper develops a sharp-minimax theory for heteroscedastic nonparametric regression, showing how the scale function influences estimation and proposing adaptive estimators that perform well even with auxiliary variables.

Contribution

It introduces a new sharp-minimax framework for heteroscedastic regression with auxiliary variables and proposes adaptive estimators that achieve optimality under these conditions.

Findings

Scale function dependence affects estimator optimality.

Proposed estimators are asymptotically sharp minimax and adaptive.

Numerical results support theoretical findings.

Abstract

The paper is devoted to the problem of estimation of a univariate component in a heteroscedastic nonparametric multiple regression under the mean integrated squared error (MISE) criteria. The aim is to understand how the scale function should be used for estimation of the univariate component. It is known that the scale function does not affect the rate of the MISE convergence, and as a result sharp constants are explored. The paper begins with developing a sharp-minimax theory for a pivotal model $Y=f(X)+\sigma(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ is standard normal and independent of the predictor X and the auxiliary vector-covariate $\mathbf{Z}$. It is shown that if the scale $\sigma(x,\mathbf{z})$ depends on the auxiliary variable, then a special estimator, which uses the scale (or its estimate), is asymptotically sharp minimax and adaptive to unknown smoothness of f(x). This is an interesting conclusion because if the scale does not depend on the auxiliary covariate $\mathbf{Z}$, then ignoring the heteroscedasticity can yield a sharp minimax estimation. The pivotal model serves as a natural benchmark for a general additive model $Y=f(X)+g(\mathbf{Z})+\sigma(X,\mathbf{Z})\varepsilon$, where $\varepsilon$ may depend on $(X,\mathbf{Z})$ and have only a finite fourth moment. It is shown that for this model a data-driven estimator can perform as well as for the benchmark. Furthermore, the estimator, suggested for continuous responses, can be also used for the case of discrete responses. Bernoulli and Poisson regressions, that are inherently heteroscedastic, are particular considered examples for which sharp minimax lower bounds are obtained as well. A numerical study shows that the asymptotic theory sheds light on small samples.