Covariance estimation for distributions with $2+\varepsilon$ moments

TL;DR

This paper determines the minimal sample size needed to accurately estimate covariance matrices for distributions with slightly more than second moments, using a novel randomized spectral sparsifier approach.

Contribution

It establishes an optimal sample size bound for distributions with bounded $2+ extvarepsilon$ moments and introduces a new randomized spectral sparsifier method for covariance estimation.

Findings

Optimal sample size N=O(n) for certain distributions

Alternative approach to the Kannan-Lovasz-Simonovits problem for log-concave distributions

Lower bounds under weaker moment assumptions

Abstract

We study the minimal sample size N=N(n) that suffices to estimate the covariance matrix of an n-dimensional distribution by the sample covariance matrix in the operator norm, with an arbitrary fixed accuracy. We establish the optimal bound N=O(n) for every distribution whose k-dimensional marginals have uniformly bounded $2+\varepsilon$ moments outside the sphere of radius $O(\sqrt{k})$. In the specific case of log-concave distributions, this result provides an alternative approach to the Kannan-Lovasz-Simonovits problem, which was recently solved by Adamczak et al. [J. Amer. Math. Soc. 23 (2010) 535-561]. Moreover, a lower estimate on the covariance matrix holds under a weaker assumption - uniformly bounded $2+\varepsilon$ moments of one-dimensional marginals. Our argument consists of randomizing the spectral sparsifier, a deterministic tool developed recently by Batson, Spielman and Srivastava [SIAM J. Comput. 41 (2012) 1704-1721]. The new randomized method allows one to control the spectral edges of the sample covariance matrix via the Stieltjes transform evaluated at carefully chosen random points.