Learning Confidence Ellipsoids and Applications to Robust Subspace Recovery

TL;DR

This paper presents a polynomial-time algorithm for finding confidence ellipsoids with volume guarantees in high-dimensional distributions, addressing computational challenges and applications to robust subspace recovery.

Contribution

It introduces the first polynomial-time method with volume approximation guarantees for confidence ellipsoids with bounded condition number in high dimensions.

Findings

Provides an $O(eta)^{ ext{poly}(d)}$ volume approximation algorithm.

Shows the algorithm covers at least $1 - O(rac{eta}{ ext{coverage}})$ probability mass.

Establishes computational hardness results indicating the necessity of the dependence on $eta$.

Abstract

We study the problem of finding confidence ellipsoids for an arbitrary distribution in high dimensions. Given samples from a distribution $D$ and a confidence parameter $\alpha$, the goal is to find the smallest volume ellipsoid $E$ which has probability mass $\mathbb{P}_{D}[E] \ge 1-\alpha$. Ellipsoids are a highly expressive class of confidence sets as they can capture correlations in the distribution, and can approximate any convex set. In statistics, this is the classic minimum volume estimator introduced by Rousseeuw as a robust non-parametric estimator of location and scatter. However in high dimensions, it becomes NP-hard to obtain any non-trivial approximation factor in volume when the condition number $\beta$ of the ellipsoid (ratio of the largest to the smallest axis length) goes to $\infty$. This motivates the focus of our paper: can we efficiently find confidence ellipsoids with volume approximation guarantees when compared to ellipsoids of bounded condition number $\beta$? Our main result is a polynomial time algorithm that finds an ellipsoid $E$ whose volume is within a $O(\beta)^{\gamma d}$ multiplicative factor of the volume of best $\beta$-conditioned ellipsoid while covering at least $1-O(\alpha/\gamma)$ probability mass for any $\gamma \in (0,1)$. In particular, setting $\gamma = o(1)$, this gives a $O(\beta)^{o(d)}$ volume approximation, with a multiplicative loss in miscoverage. We complement this with a computational hardness result that shows that such a dependence on $\beta$ seems necessary, even with some slack in coverage. The algorithm and analysis uses the rich primal-dual structure of the minimum volume enclosing ellipsoid and the geometric Brascamp-Lieb inequality. As a consequence, we obtain the first polynomial time algorithm with approximation guarantees on worst-case instances of the robust subspace recovery problem.