Matrix Moment and Concentration Inequalities for Martingales and Ergodic Markov Chains with Applications in Statistical Learning

TL;DR

This paper develops new spectral norm concentration inequalities for dependent random matrices, especially Markov chains, with applications in statistical learning, improving upon prior results by assuming geometric ergodicity and achieving dimension-free bounds.

Contribution

The paper introduces novel matrix inequalities for dependent data under geometric ergodicity, matching CLT variance terms, and extends results to infinite-dimensional operators with dimension-free bounds.

Findings

Established matrix Rosenthal, Hoeffding, Bernstein inequalities for Markov chains.

Provided dimension-free inequalities based on effective rank.

Applied results to covariance estimation and PCA on Markovian data.

Abstract

In this paper, we study moment and concentration inequalities for the spectral norm of sums of dependent random matrices. We establish novel Rosenthal-Burkholder inequalities for discrete-time matrix local martingales, Burkholder-Davis-Gundy inequality for continuous matrix local martingales, as well as matrix Rosenthal, Hoeffding, and Bernstein inequalities for ergodic Markov chains. Compared with previous work on matrix concentration inequalities for Markov chains, which assume a non-zero absolute $L^2$-spectral gap or the stronger $\psi$-mixing condition, our results assume geometric ergodicity, a condition commonly used in statistical applications. Furthermore, our results have leading terms that match the Markov chain central limit theorem, rather than relying on suboptimal variance proxies. We also give dimension-free versions of the inequalities, which are independent of the ambient dimension $d$ and relies on the effective rank instead. This enables the generalization of our results to linear operators in infinite-dimensional Hilbert spaces. Our results have extensive applications in statistics and machine learning; in particular, we obtain improved bounds in covariance estimation and principal component analysis on Markovian data.