Robust Local Polynomial Regression with Similarity Kernels

TL;DR

This paper introduces a robust local polynomial regression method that uses a novel similarity kernel incorporating both predictor and response variables, significantly improving robustness against outliers and noise.

Contribution

It proposes a new conditional density kernel for local polynomial regression that enhances robustness and accuracy without iterative procedures.

Findings

Improves robustness and accuracy in noisy, heteroscedastic data

Outperforms standard LPR in synthetic benchmarks

Available implementation demonstrates practical utility

Abstract

Local Polynomial Regression (LPR) is a widely used nonparametric method for modeling complex relationships due to its flexibility and simplicity. It estimates a regression function by fitting low-degree polynomials to localized subsets of the data, weighted by proximity. However, traditional LPR is sensitive to outliers and high-leverage points, which can significantly affect estimation accuracy. This paper revisits the kernel function used to compute regression weights and proposes a novel framework that incorporates both predictor and response variables in the weighting mechanism. The focus of this work is a conditional density kernel that robustly estimates weights by mitigating the influence of outliers through localized density estimation. A related joint density kernel is also discussed in an appendix. The proposed method is implemented in Python and is publicly available at https://github.com/yaniv-shulman/rsklpr, demonstrating competitive performance in synthetic benchmark experiments. Compared to standard LPR, the proposed approach consistently improves robustness and accuracy, especially in heteroscedastic and noisy environments, without requiring multiple iterations. This advancement provides a promising extension to traditional LPR, opening new possibilities for robust regression applications.