Optimal estimation of variance in nonparametric regression with random design

TL;DR

This paper derives the optimal rates for estimating variance functions in heteroscedastic nonparametric regression with random design, extending previous fixed design results and proposing a U-statistic-based estimator.

Contribution

It establishes the minimax rates for variance estimation under both local and global risks in a heteroscedastic nonparametric regression with random design, extending fixed design results.

Findings

Minimax rate of variance estimation is n^{-rac{8etaeta}{4etaeta + 2eta + eta}} ext{ or } n^{-rac{2eta}{2eta+1}}.

Special case of constant variance yields a rate of n^{-8eta/(4eta+1)} ext{ or } n^{-1}.

Proposes a U-statistic-based local polynomial estimator that achieves the minimax rate.

Abstract

Consider the heteroscedastic nonparametric regression model with random design \begin{align*} Y_i = f(X_i) + V^{1/2}(X_i)\varepsilon_i, \quad i=1,2,\ldots,n, \end{align*} with $f(\cdot)$ and $V(\cdot)$ $\alpha$- and $\beta$-H\"older smooth, respectively. We show that the minimax rate of estimating $V(\cdot)$ under both local and global squared risks is of the order \begin{align*} n^{-\frac{8\alpha\beta}{4\alpha\beta + 2\alpha + \beta}} \vee n^{-\frac{2\beta}{2\beta+1}}, \end{align*} where $a\vee b := \max\{a,b\}$ for any two real numbers $a,b$. This result extends the fixed design rate $n^{-4\alpha} \vee n^{-2\beta/(2\beta+1)}$ derived in Wang et al. [2008] in a non-trivial manner, as indicated by the appearances of both $\alpha$ and $\beta$ in the first term. In the special case of constant variance, we show that the minimax rate is $n^{-8\alpha/(4\alpha+1)}\vee n^{-1}$ for variance estimation, which further implies the same rate for quadratic functional estimation and thus unifies the minimax rate under the nonparametric regression model with those under the density model and the white noise model. To achieve the minimax rate, we develop a U-statistic-based local polynomial estimator and a lower bound that is constructed over a specified distribution family of randomness designed for both $\varepsilon_i$ and $X_i$.