Dimension-free Bounds for Sum of Dependent Matrices and Operators with Heavy-Tailed Distribution

TL;DR

This paper establishes dimension-free deviation bounds for sums of dependent, heavy-tailed high-dimensional matrices, extending existing results and applicable to covariance estimation, hidden Markov models, and linear regression.

Contribution

It introduces a novel dimension-free bound for dependent heavy-tailed matrices using variational approximation and eigenvalue truncation techniques.

Findings

Bound depends on effective rank, not dimension

Applicable to covariance matrices, HMMs, and linear regression

Corrects previous theorem with a new log-Sobolev inequality

Abstract

We prove deviation inequalities for sums of high-dimensional random matrices and operators with dependence and {\rc heavy tails}. Estimation of high-dimensional matrices is a concern for numerous modern applications. However, most results are stated for independent observations. Therefore, it is critical to derive results for dependent and heavy-tailed matrices. In this paper, we derive a dimension-free upper bound on the deviation of the sums. Thus, the bound does not depend explicitly on the dimension of the matrices but rather on their effective rank. Our result generalizes several existing studies on the deviation of sums of matrices. It relies on two techniques: (i) a variational approximation of the dual of moment generating functions, and (ii) robustification through the truncation of the eigenvalues of the matrices. We reveal that our results are applicable to several problems, such as covariance matrix estimation, hidden Markov models, and overparameterized linear regression. At the beginning, we have attached a corrigendum of the original paper. We correct Theorem 4 of the original paper by introducing a log-Sobolev inequality in place of the boundedness condition. We show that the examples discussed in the original paper can be recovered under new conditions. The original paper, uncorrected version -- which includes the aforementioned error -- is appended after this corrigendum for transparency and comparison.