Sharp Matrix Empirical Bernstein Inequalities

TL;DR

This paper introduces two sharp empirical Bernstein inequalities for symmetric random matrices with bounded eigenvalues, which adapt to unknown variance and match classical bounds asymptotically, applicable to independent and martingale-dependent data.

Contribution

The paper develops two novel empirical Bernstein inequalities for matrices that are sharp, adaptive to variance, and applicable under different dependence structures.

Findings

Inequalities are tight and adapt to unknown variance.

First inequality applies to independent matrix samples.

Second inequality applies under martingale dependence at stopping times.

Abstract

We present two sharp, closed-form empirical Bernstein inequalities for symmetric random matrices with bounded eigenvalues. By sharp, we mean that both inequalities adapt to the unknown variance in a tight manner: the deviation captured by the first-order $1/\sqrt{n}$ term asymptotically matches the matrix Bernstein inequality exactly, including constants, the latter requiring knowledge of the variance. Our first inequality holds for the sample mean of independent matrices, and our second inequality holds for a mean estimator under martingale dependence at stopping times.