Almost sharp covariance and Wishart-type matrix estimation

TL;DR

This paper provides precise bounds on the deviation of covariance matrix estimators for Gaussian vectors with independent entries, improving existing results by analyzing the structure of the problem and capturing fourth-moment effects.

Contribution

It introduces a refined analysis of covariance deviation bounds for Gaussian vectors with independent entries, incorporating fourth-moment dependencies and providing improved bounds and matching lower bounds.

Findings

Derived explicit bounds on covariance deviation with moment considerations.

Showed improvement over previous bounds in specific examples.

Provided lower bounds matching the derived upper bounds.

Abstract

Let $X_1,..., X_n \in \mathbb{R}^d$ be independent Gaussian random vectors with independent entries and variance profile $(b_{ij})_{i \in [d],j \in [n]}$. A major question in the study of covariance estimation is to give precise control on the deviation of $\sum_{j \in [n]}X_jX_j^T-\mathbb{E} X_jX_j^T$. We show that under mild conditions, we have \begin{align*} \mathbb{E} \left\|\sum_{j \in [n]}X_jX_j^T-\mathbb{E} X_jX_j^T\right\| \lesssim \max_{i \in [d]}\left(\sum_{j \in [n]}\sum_{l \in [d]}b_{ij}^2b_{lj}^2\right)^{1/2}+\max_{j \in [n]}\sum_{i \in [d]}b_{ij}^2+\text{error}. \end{align*} The error is quantifiable, and we often capture the $4$th-moment dependency already presented in the literature for some examples. The proofs are based on the moment method and a careful analysis of the structure of the shapes that matter. We also provide examples showing improvement over the past works and matching lower bounds.