Do we need to estimate the variance in robust mean estimation?

TL;DR

This paper introduces self-tuned robust mean estimators for heavy-tailed distributions that adaptively estimate variance, achieving optimal finite-sample performance without cross-validation.

Contribution

The paper presents a novel loss function and joint optimization approach for automatically tuning the robustification parameter in mean estimation.

Findings

Estimator achieves the Cramér-Rao lower bound.

Outperforms previous methods in efficiency.

Does not require cross-validation or Lepski's method.

Abstract

In this paper, we propose self-tuned robust estimators for estimating the mean of heavy-tailed distributions, which refer to distributions with only finite variances. Our approach introduces a new loss function that considers both the mean parameter and a robustification parameter. By jointly optimizing the empirical loss function with respect to both parameters, the robustification parameter estimator can automatically adapt to the unknown data variance, and thus the self-tuned mean estimator can achieve optimal finite-sample performance. Our method outperforms previous approaches in terms of both computational and asymptotic efficiency. Specifically, it does not require cross-validation or Lepski's method to tune the robustification parameter, and the variance of our estimator achieves the Cram\'er-Rao lower bound. Project source code is available at \url{https://github.com/statsle/automean}.