# Concentration inequalities for random matrix products

## TL;DR

This paper establishes sharp nonasymptotic concentration inequalities for normalized products of independent bounded random matrices, with applications to stochastic algorithms like streaming PCA.

## Contribution

It provides the first nonasymptotic spectral norm bounds for a broad class of random matrix products, combining matrix Bernstein inequality and combinatorial methods.

## Key findings

- Spectral norm error bound of O((log n)^2 log(d/δ)/√n) with high probability
- Convergence of normalized matrix products to matrix exponential e^{X}
- Sharpness of the rate up to logarithmic factors

## Abstract

Suppose $\{ X_k \}_{k \in \mathbb{Z}}$ is a sequence of bounded independent random matrices with common dimension $d\times d$ and common expectation $\mathbb{E}[ X_k ]= X$. Under these general assumptions, the normalized random matrix product $$Z_n = (I + \frac{1}{n}X_n)(I + \frac{1}{n}X_{n-1}) \cdots (I + \frac{1}{n}X_1)$$ converges to $Z_n \rightarrow e^{X}$ as $n \rightarrow \infty$. Normalized random matrix products of this form arise naturally in stochastic iterative algorithms, such as Oja's algorithm for streaming Principal Component Analysis. Here, we derive nonasymptotic concentration inequalities for such random matrix products. In particular, we show that the spectral norm error satisfies $\| Z_n - e^{X} \| = O((\log(n))^2\log(d/\delta)/\sqrt{n})$ with probability exceeding $1-\delta$. This rate is sharp in $n$, $d$, and $\delta$, up to possibly the $\log(n)$ and $\log(d)$ factors. The proof relies on two key points of theory: the Matrix Bernstein inequality concerning the concentration of sums of random matrices, and Baranyai's theorem from combinatorial mathematics. Concentration bounds for general classes of random matrix products are hard to come by in the literature, and we hope that our result will inspire further work in this direction.