Closed-form empirical Bernstein confidence sequences for scalars and matrices

TL;DR

This paper introduces a new closed-form, variance-adaptive confidence sequence for scalar and matrix-valued data, offering tighter bounds and desirable properties for tracking means over large sample sizes.

Contribution

It presents the first closed-form, variance-adaptive confidence sequence for both scalars and matrices, improving tightness and asymptotic properties over existing methods.

Findings

Yields the tightest closed-form CS for sample sizes up to 10^6

Asymptotically tighter than previous CS when means are constant

Has limiting width independent of significance level

Abstract

We derive a new closed-form variance-adaptive confidence sequence (CS) for estimating the average conditional mean of a sequence of bounded random variables. Empirically, it yields the tightest closed-form CS we have found for tracking time-varying means, across sample sizes up to $\approx 10^6$. When the observations happen to have the same conditional mean, our CS is asymptotically tighter than the recent closed-form CS of Waudby-Smith and Ramdas [38]. It also has other desirable properties: it is centered at the unweighted sample mean and has limiting width (multiplied by $\sqrt{t/\log t}$) independent of the significance level. We extend our results to provide a CS with the same properties for random matrices with bounded eigenvalues.