High-Dimensional Robust Mean Estimation with Untrusted Batches

TL;DR

This paper develops algorithms for high-dimensional mean estimation in a setting with untrusted data batches, addressing both adversarial users and distributional heterogeneity, and achieves minimax-optimal error bounds.

Contribution

It introduces two Sum-of-Squares based algorithms that handle tiered corruption in high-dimensional, multi-user environments with provably optimal error rates.

Findings

Achieves minimax-optimal error rate of O(√(ε/n) + √(d/nN) + √α)

Demonstrates suppression of adversarial influence by batch averaging

Addresses new challenges in high-dimensional, sample-level corruption scenarios

Abstract

We study high-dimensional mean estimation in a collaborative setting where data is contributed by $N$ users in batches of size $n$. In this environment, a learner seeks to recover the mean $\mu$ of a true distribution $P$ from a collection of sources that are both statistically heterogeneous and potentially malicious. We formalize this challenge through a double corruption landscape: an $\varepsilon$-fraction of users are entirely adversarial, while the remaining ``good'' users provide data from distributions that are related to $P$, but deviate by a proximity parameter $\alpha$. Unlike existing work on the untrusted batch model, which typically measures this deviation via total variation distance in discrete settings, we address the continuous, high-dimensional regime under two natural variants for deviation: (1) good batches are drawn from distributions with a mean-shift of $\sqrt{\alpha}$, or (2) an $\alpha$-fraction of samples within each good batch are adversarially corrupted. In particular, the second model presents significant new challenges: in high dimensions, unlike discrete settings, even a small fraction of sample-level corruption can shift empirical means and covariances arbitrarily. We provide two Sum-of-Squares (SoS) based algorithms to navigate this tiered corruption. Our algorithms achieve the minimax-optimal error rate $O(\sqrt{\varepsilon/n} + \sqrt{d/nN} + \sqrt{\alpha})$, demonstrating that while heterogeneity $\alpha$ represents an inherent statistical difficulty, the influence of adversarial users is suppressed by a factor of $1/\sqrt{n}$ due to the internal averaging afforded by the batch structure.