Robust nonparametric regression: review and practical considerations

TL;DR

This paper reviews robust nonparametric regression methods, focusing on outlier resistance, practical issues like bandwidth selection, and scalability with many covariates, providing guidance for real-world applications.

Contribution

It offers a comprehensive review of outlier robust estimation techniques, bandwidth selection methods, and scalable approaches for high-dimensional nonparametric regression.

Findings

Robust methods improve resistance to outliers in nonparametric regression.

Discussion of practical bandwidth selection techniques for robust estimators.

Recent scalable methods enable nonparametric regression with many covariates.

Abstract

Nonparametric regression models offer a way to understand and quantify relationships between variables without having to identify an appropriate family of possible regression functions. Although many estimation methods for these models have been proposed in the literature, most of them can be highly sensitive to the presence of a small proportion of atypical observations in the training set. In this paper we review outlier robust estimation methods for nonparametric regression models, paying particular attention to practical considerations. Since outliers can also influence negatively the regression estimator by affecting the selection of bandwidths or smoothing parameters, we also discuss available robust alternatives for this task. Finally, since using many of the ``classical'' nonparametric regression estimators (and their robust counterparts) can be very challenging in settings with a moderate or large number of explanatory variables, we review recent robust nonparametric regression methods that scale well with a growing number of covariates.