Empirical Bernstein in smooth Banach spaces
Diego Martinez-Taboada, Aaditya Ramdas

TL;DR
This paper introduces a new empirical Bernstein inequality for vector-valued random variables in smooth Banach spaces, enabling tighter confidence bounds using empirical variance estimates in both batch and sequential settings.
Contribution
It derives the first empirical Bernstein inequality applicable in 2-smooth Banach spaces, replacing true variance with empirical estimates for improved confidence bounds.
Findings
Confidence sets match Bernstein bounds asymptotically
Applicable to both batch and sequential sampling
Includes finite-dimensional Euclidean and Hilbert spaces
Abstract
Existing concentration bounds for bounded vector-valued random variables include extensions of the scalar Hoeffding and Bernstein inequalities. While the latter is typically tighter, it requires knowing a bound on the variance of the random variables. We derive a new vector-valued empirical Bernstein inequality, which makes use of an empirical estimator of the variance instead of the true variance. The bound holds in 2-smooth separable Banach spaces, which include finite dimensional Euclidean spaces and separable Hilbert spaces. The resulting confidence sets are instantiated for both the batch setting (where the sample size is fixed) and the sequential setting (where the sample size is a stopping time). The confidence set width asymptotically exactly matches that achieved by Bernstein in the leading term. The method and supermartingale proof technique combine several tools of Pinelis…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsApproximation Theory and Sequence Spaces · Advanced Banach Space Theory
