Anisotropic local law for non-separable sample covariance matrices

TL;DR

This paper proves optimal local laws for non-separable sample covariance matrices, extending results beyond traditional models to dependent and nonlinearly transformed data, with applications in machine learning and statistics.

Contribution

It introduces a tensor network framework and establishes both averaged and anisotropic local laws for complex covariance models with dependent entries.

Findings

Proves optimal averaged local law for non-separable covariance matrices.

Establishes full anisotropic local law under structural cumulant assumptions.

Applies results to models in machine learning and nonlinear data transformations.

Abstract

We establish local laws for sample covariance matrices $K = N^{-1}\sum_{i=1}^N \g_i\g_i^*$ where the random vectors $\g_1, \ldots, \g_N \in \R^n$ are independent with common covariance $\Sigma$. Previous work has largely focused on the separable model $\g = \Sigma^{1/2}\w$ with $\w$ having independent entries, but this structure is rarely present in statistical applications involving dependent or nonlinearly transformed data. Under a concentration assumption for quadratic forms $\g^*A\g$, we prove an optimal averaged local law showing that the Stieltjes transform of $K$ converges to its deterministic limit uniformly down to the optimal scale $\eta \geq N^{-1+\eps}$. Under an additional structural assumption on the cumulant tensors of $\g$ -- which interpolates between the highly structured case of independent entries and generic dependence -- we establish the full anisotropic local law, providing entrywise control of the resolvent $(K-zI)^{-1}$ in arbitrary directions. We discuss several classes of non-separable examples satisfying our assumptions, including conditionally mean-zero distributions, the random features model $\g = \sigma(X\w)$ arising in machine learning, and Gaussian measures with nonlinear tilting. The proofs introduce a tensor network framework for analyzing fluctuation averaging in the presence of higher-order cumulant structure.