Sample covariance matrices of heavy-tailed distributions

TL;DR

This paper establishes bounds on the deviation of sample covariance matrices from the true covariance for heavy-tailed distributions, extending Bai-Yin type results to distributions with finite moments of order greater than four.

Contribution

It provides new probabilistic bounds for the spectral norm deviation of sample covariance matrices under heavy-tailed assumptions, generalizing classical results.

Findings

Bounds depend on the number of samples and dimension

Results hold for distributions with finite p-th moments, p>4

Quantitative Bai-Yin type theorem derived

Abstract

Let $p>2$, $B\geq 1$, $N\geq n$ and let $X$ be a centered $n$-dimensional random vector with the identity covariance matrix such that $\sup\limits_{a\in S^{n-1}}{\mathrm E}|\langle X,a\rangle|^p\leq B$. Further, let $X_1,X_2,\dots,X_N$ be independent copies of $X$, and $\Sigma_N:=\frac{1}{N}\sum_{i=1}^N X_i {X_i}^T$ be the sample covariance matrix. We prove that $$K^{-1}\|\Sigma_N-I_n\|_{2\to 2}\leq\frac{1}{N}\max\limits_{i\leq N}\|X_i\|^2 +\Bigl(\frac{n}{N}\Bigr)^{1-2/p}\log^4\frac{N}{n}+\Bigl(\frac{n}{N}\Bigr)^{1-2/\min(p,4)}$$ with probability at least $1-\frac{1}{n}$, where $K>0$ depends only on $B$ and $p$. In particular, for all $p>4$ we obtain a quantitative Bai-Yin type theorem.