A robust, adaptive M-estimator for pointwise estimation in heteroscedastic regression

TL;DR

This paper presents a fully adaptive, robust M-estimator for pointwise estimation in heteroscedastic regression that adapts to unknown noise, design distributions, and function smoothness, even under contamination and outliers.

Contribution

It introduces a new adaptive M-estimator that does not require moment conditions or positive density assumptions, with a data-driven bandwidth selection for enhanced robustness and adaptivity.

Findings

The estimator is consistent under weak assumptions.

It achieves minimax optimality over H"older spaces.

It adapts to noise, contamination, outliers, and function smoothness.

Abstract

We introduce a robust and fully adaptive method for pointwise estimation in heteroscedastic regression. We allow for noise and design distributions that are unknown and fulfill very weak assumptions only. In particular, we do not impose moment conditions on the noise distribution. Moreover, we do not require a positive density for the design distribution. In a first step, we study the consistency of locally polynomial M-estimators that consist of a contrast and a kernel. Afterwards, minimax results are established over unidimensional H\"older spaces for degenerate design. We then choose the contrast and the kernel that minimize an empirical variance term and demonstrate that the corresponding M-estimator is adaptive with respect to the noise and design distributions and adaptive (Huber) minimax for contamination models. In a second step, we additionally choose a data-driven bandwidth via Lepski's method. This leads to an M-estimator that is adaptive with respect to the noise and design distributions and, additionally, adaptive with respect to the smoothness of an isotropic, multivariate, locally polynomial target function. These results are also extended to anisotropic, locally constant target functions. Our data-driven approach provides, in particular, a level of robustness that adapts to the noise, contamination, and outliers.