Outrigger local polynomial regression

TL;DR

This paper introduces the outrigger local polynomial estimator, a method that adapts to various error distributions in nonparametric regression, achieving near-minimax optimality without strict assumptions on errors or covariates.

Contribution

The paper proposes the outrigger estimator, which uses a conditional score estimate and a broader local window to improve distributional adaptivity in local polynomial regression.

Findings

Achieves asymptotic risk ratio at most 1 compared to the Gaussian case.

Proven minimax optimality over Hölder classes up to a constant factor.

Validated through simulations and real data, available in R implementation.

Abstract

Standard local polynomial estimators of a nonparametric regression function employ a weighted least squares loss function that is tailored to the setting of homoscedastic Gaussian errors. We introduce the outrigger local polynomial estimator, which is designed to achieve distributional adaptivity across different conditional error distributions. It modifies a standard local polynomial estimator by employing an estimate of the conditional score function of the errors and an 'outrigger' that draws on the data in a broader local window to stabilise the influence of the conditional score estimate. Subject to smoothness and moment conditions, and only requiring consistency of the conditional score estimate, we first establish that even under the least favourable settings for the outrigger estimator, the asymptotic ratio of the worst-case local risks of the two estimators is at most $1$, with equality if and only if the conditional error distribution is Gaussian. Moreover, we prove that the outrigger estimator is minimax optimal over H\"older classes up to a multiplicative factor $A_{\beta,d}$, depending only on the smoothness $\beta \in (0,\infty)$ of the regression function and the dimension~$d$ of the covariates. When $\beta \in (0,1]$, we find that $A_{\beta,d} \leq 1.69$, with $\lim_{\beta \searrow 0} A_{\beta,d} = 1$. A further attraction of our proposal is that we do not require structural assumptions such as independence of errors and covariates, or symmetry of the conditional error distribution. Numerical results on simulated and real data validate our theoretical findings; our methodology is implemented in R and available at https://github.com/elliot-young/outrigger.