Robust Kernel Density Estimation

TL;DR

This paper introduces a robust kernel density estimator (RKDE) that combines traditional KDE with M-estimation to improve robustness against outliers, supported by theoretical analysis and experimental validation.

Contribution

It presents a novel robust KDE method using M-estimation, with an efficient IRWLS algorithm and theoretical convergence guarantees.

Findings

RKDE is robust to contamination and outliers.

The method is computationally efficient via IRWLS.

Experimental results show improved density estimation and anomaly detection.

Abstract

We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical $M$-estimation. We interpret the KDE based on a radial, positive semi-definite kernel as a sample mean in the associated reproducing kernel Hilbert space. Since the sample mean is sensitive to outliers, we estimate it robustly via $M$-estimation, yielding a robust kernel density estimator (RKDE). An RKDE can be computed efficiently via a kernelized iteratively re-weighted least squares (IRWLS) algorithm. Necessary and sufficient conditions are given for kernelized IRWLS to converge to the global minimizer of the $M$-estimator objective function. The robustness of the RKDE is demonstrated with a representer theorem, the influence function, and experimental results for density estimation and anomaly detection.