Concentration study of M-estimators using the influence function

TL;DR

This paper introduces a new finite-sample analysis of M-estimators using influence functions, providing robust, high-dimensional mean estimation with optimal convergence rates and computational efficiency.

Contribution

It offers a novel influence function-based approach for analyzing M-estimators, enabling robust high-dimensional mean estimation with provable guarantees and practical computational advantages.

Findings

Achieves minimax convergence rate in high-dimensional robust mean estimation.

Provides a computationally efficient estimator with $O(nd\log(Tr(\Sigma)))$ complexity.

Demonstrates practical speed and robustness in heavy-tailed and corrupted data settings.

Abstract

We present a new finite-sample analysis of M-estimators of locations in $\mathbb{R}^d$ using the tool of the influence function. In particular, we show that the deviations of an M-estimator can be controlled thanks to its influence function (or its score function) and then, we use concentration inequality on M-estimators to investigate the robust estimation of the mean in high dimension in a corrupted setting (adversarial corruption setting) for bounded and unbounded score functions. For a sample of size $n$ and covariance matrix $\Sigma$, we attain the minimax speed $\sqrt{Tr(\Sigma)/n}+\sqrt{\|\Sigma\|_{op}\log(1/\delta)/n}$ with probability larger than $1-\delta$ in a heavy-tailed setting. One of the major advantages of our approach compared to others recently proposed is that our estimator is tractable and fast to compute even in very high dimension with a complexity of $O(nd\log(Tr(\Sigma)))$ where $n$ is the sample size and $\Sigma$ is the covariance matrix of the inliers. In practice, the code that we make available for this article proves to be very fast.