Debiased Nonparametric Regression for Statistical Inference and Distributionally Robustness

TL;DR

This paper introduces a model-free debiasing technique for nonparametric regression estimators, ensuring their statistical validity and robustness, which broadens their applicability in inference and distributionally robust settings.

Contribution

The paper presents a novel debiasing method that guarantees risk convergence and asymptotic normality for nonparametric estimators, addressing a gap in theoretical understanding.

Findings

Debiased estimators achieve pointwise and uniform risk convergence.

The method ensures asymptotic normality under mild conditions.

Enhanced robustness to covariate shift in nonparametric regression.

Abstract

This study proposes a debiasing method for smooth nonparametric estimators. While machine learning techniques such as random forests and neural networks have demonstrated strong predictive performance, their theoretical properties remain relatively underexplored. In particular, many modern algorithms lack guarantees of pointwise and uniform risk convergence, as well as asymptotic normality. These properties are essential for statistical inference and robust estimation and have been well-established for classical methods such as Nadaraya-Watson regression. To ensure these properties for various nonparametric regression estimators, we introduce a model-free debiasing method. By incorporating a correction term that estimates the conditional expected residual of the original estimator, or equivalently, its estimation error, into the initial nonparametric regression estimator, we obtain a debiased estimator that satisfies pointwise and uniform risk convergence, along with asymptotic normality, under mild smoothness conditions. These properties facilitate statistical inference and enhance robustness to covariate shift, making the method broadly applicable to a wide range of nonparametric regression problems.