High-Accuracy List-Decodable Mean Estimation

TL;DR

This paper introduces a new approach to list-decodable mean estimation that achieves high accuracy with a manageable list size, providing both theoretical guarantees and an efficient algorithm for Gaussian distributions.

Contribution

It presents the first high-accuracy list-decodable mean estimation method with explicit bounds and a novel algorithmic approach avoiding sum-of-squares techniques.

Findings

Existence of a list with size exponential in rac{lpha}{\u001epsilon^2} where one element is within psilon of the true mean.

Algorithm achieves runtime and sample complexity with double exponential dependence on rac{lpha}{psilon^2}.

New proof of identifiability and a novel algorithmic framework independent of sum-of-squares hierarchy.

Abstract

In list-decodable learning, we are given a set of data points such that an $\alpha$-fraction of these points come from a nice distribution $D$, for some small $\alpha \ll 1$, and the goal is to output a short list of candidate solutions, such that at least one element of this list recovers some non-trivial information about $D$. By now, there is a large body of work on this topic; however, while many algorithms can achieve optimal list size in terms of $\alpha$, all known algorithms must incur error which decays, in some cases quite poorly, with $1 / \alpha$. In this paper, we ask if this is inherent: is it possible to trade off list size with accuracy in list-decodable learning? More formally, given $\epsilon > 0$, can we can output a slightly larger list in terms of $\alpha$ and $\epsilon$, but so that one element of this list has error at most $\epsilon$ with the ground truth? We call this problem high-accuracy list-decodable learning. Our main result is that non-trivial high-accuracy guarantees, both information-theoretically and algorithmically, are possible for the canonical setting of list-decodable mean estimation of identity-covariance Gaussians. Specifically, we demonstrate that there exists a list of candidate means of size at most $L = \exp \left( O\left( \tfrac{\log^2 1 / \alpha}{\epsilon^2} \right)\right)$ so that one of the elements of this list has $\ell_2$ distance at most $\epsilon$ to the true mean. We also design an algorithm that outputs such a list with runtime and sample complexity $n = d^{O(\log L)} + \exp \exp (\widetilde{O}(\log L))$. We do so by demonstrating a completely novel proof of identifiability, as well as a new algorithmic way of leveraging this proof without the sum-of-squares hierarchy, which may be of independent technical interest.