A generalized Catoni's ${\rm M}$-estimator under finite {$\alpha$-th moment assumption} with $\alpha \in (1,2)$

TL;DR

This paper extends Catoni's M-estimator to cases where data have finite lpha-th moments with lpha in (1,2), providing improved robustness and deviation bounds under weaker assumptions than finite variance.

Contribution

The authors develop a generalized M-estimator for lpha-th moments, modifying the influence function to handle data with only finite lpha-th moments, and demonstrate its superior performance over empirical mean.

Findings

The generalized estimator achieves deviation bounds comparable to previous methods as lpha approaches 2.

Experimental results show improved performance over empirical mean, especially for smaller lpha.

Application to  regression under relaxed moment assumptions demonstrates the estimator's practical utility.

Abstract

We generalize the { ${\rm M}$-estimator} put forward by Catoni in his seminal paper [C12] to the case in which samples can have finite $\alpha$-th moment with $\alpha \in (1,2)$ rather than finite variance, our approach is by slightly modifying the influence function $\varphi$ therein. The choice of the new influence function is inspired by the Taylor-like expansion developed in [C-N-X]. We obtain a deviation bound of the estimator, as $\alpha \rightarrow 2$, this bound is the same as that in [C12]. Experiment shows that our generalized ${\rm M}$-estimator performs better than the empirical mean estimator, the smaller the $\alpha$ is, the better the performance will be. As an application, we study an $\ell_{1}$ regression considered by Zhang et al. [Z-Z] who assumed that samples have finite variance, and relax their assumption to be finite {$\alpha$-th} moment with $\alpha \in (1,2)$.