Reducing variance in univariate smoothing

TL;DR

This paper introduces a variance reduction method for univariate smoothing that improves efficiency by combining estimators at nearby points without increasing bias, applicable to local linear regression.

Contribution

It proposes a novel variance reduction technique that maintains bias while reducing variance in nonparametric smoothing, with detailed analysis for univariate local linear regression.

Findings

The new estimator achieves appealing asymptotic relative efficiencies.

Bandwidth selection can be easily adjusted from local linear estimation rules.

Finite sample efficiency often matches asymptotic efficiency in simulations.

Abstract

A variance reduction technique in nonparametric smoothing is proposed: at each point of estimation, form a linear combination of a preliminary estimator evaluated at nearby points with the coefficients specified so that the asymptotic bias remains unchanged. The nearby points are chosen to maximize the variance reduction. We study in detail the case of univariate local linear regression. While the new estimator retains many advantages of the local linear estimator, it has appealing asymptotic relative efficiencies. Bandwidth selection rules are available by a simple constant factor adjustment of those for local linear estimation. A simulation study indicates that the finite sample relative efficiency often matches the asymptotic relative efficiency for moderate sample sizes. This technique is very general and has a wide range of applications.