Quantitative deterministic equivalent of sample covariance matrices with a general dependence structure

TL;DR

This paper extends the deterministic equivalent analysis of sample covariance matrices with dependent structures, providing quantitative bounds and applications to spectral distribution convergence and machine learning regularization.

Contribution

It introduces new bounds for the resolvent of covariance matrices with general dependence, allowing spectral parameters closer to the real axis, and applies these results to spectral convergence and machine learning.

Findings

Quantitative bounds for resolvent with dependence structures

Improved convergence rates for empirical spectral distributions

Application to regularization in Random Features models

Abstract

We study sample covariance matrices arising from rectangular random matrices with i.i.d. columns. It was previously known that the resolvent of these matrices admits a deterministic equivalent when the spectral parameter stays bounded away from the real axis. We extend this work by proving quantitative bounds involving both the dimensions and the spectral parameter, in particular allowing it to get closer to the real positive semi-line. As applications, we obtain a new bound for the convergence in Kolmogorov distance of the empirical spectral distributions of these general models. We also apply our framework to the problem of regularization of Random Features models in Machine Learning without Gaussian hypothesis.