Robust Statistical Estimators with Bounded Empirical Sensitivity

TL;DR

This paper introduces empirical sensitivity as a robustness measure for statistical estimators and analyzes its bounds in Gaussian mean estimation, revealing fundamental limitations and near-optimal bounds.

Contribution

The paper defines empirical sensitivity, establishes lower bounds for Gaussian mean estimators, and demonstrates these bounds are tight up to logarithmic factors.

Findings

Lower bounds on empirical sensitivity for optimal Gaussian mean estimators

Empirical sensitivity is at least . . . for estimators with optimal error bounds

Bounds are nearly tight, matching recent robust mean estimation results

Abstract

We introduce a new measure of robustness for statistical estimators, which we call \emph{empirical sensitivity}. An estimator $\hat \theta$ has bounded empirical sensitivity if, with high probability over a dataset $X = (X_1, \dots, X_n) \sim \mathcal{D}^{\otimes n}$, for any dataset $Y$ obtained by modifying at most $\eta n$ points in $X$, we have that $\hat \theta(Y)$ is close to $\hat \theta(X)$. We study bounds on this quantity for the prototypical problem of Gaussian mean estimation. We prove new lower bounds, showing that for any estimator $\hat \mu$ which achieves an optimal $\ell_2$-error bound of $O\left(\sqrt{d/n}\right)$, the empirical sensitivity is at least $\Omega\left(\eta + \sqrt{\eta d/n}\right)$. The two terms arise due to obstructions on the mean and variance (via an Efron-Stein argument) of such an estimator. We show that this bound is tight up to logarithmic factors, by employing recent results for robust empirical mean estimation.