Sharper dimension-free bounds on the Frobenius distance between sample covariance and its expectation

TL;DR

This paper provides improved, dimension-free probabilistic bounds on the Frobenius norm difference between sample covariance matrices and their true expectation, especially for matrices with moderate effective rank.

Contribution

It introduces sharper, dimension-free bounds on the Frobenius distance, enhancing understanding of covariance estimation accuracy in high dimensions.

Findings

Bound on Frobenius norm difference with high probability

Difference is at most proportional to trace of Sigma squared over n

Applicable to matrices with moderate effective rank

Abstract

We study properties of a sample covariance estimate $\widehat \Sigma$ given a finite sample of $n$ i.i.d. centered random elements in $\R^d$ with the covariance matrix $\Sigma$. We derive dimension-free bounds on the squared Frobenius norm of $(\widehat\Sigma - \Sigma)$ under reasonable assumptions. For instance, we show that $\smash{\|\widehat\Sigma - \Sigma\|_{\rm F}^2}$ differs from its expectation by at most $\smash{\mathcal O({\rm{Tr}}(\Sigma^2) / n)}$ with overwhelming probability, which is a significant improvement over the existing results. This allows us to establish the concentration phenomenon for the squared Frobenius distance between the covariance and its empirical counterpart in the case of moderately large effective rank of $\Sigma$.