Time-Uniform Confidence Spheres for Means of Random Vectors

TL;DR

This paper develops time-uniform confidence spheres for estimating the mean of random vectors in various distributional settings, including sub-Gaussian, log-concave, and heavy-tailed, with many results being optimal and applicable under martingale assumptions.

Contribution

It introduces a suite of dimension-free, time-uniform confidence spheres for multivariate means across diverse distribution classes, extending existing methods to non-i.i.d. and heavy-tailed data.

Findings

Dimension-free CSSs for log-concave and sub-Gaussian vectors

CSSs that adapt to time-varying means using Robbins' approach

CSSs applicable to heavy-tailed vectors with minimal moments

Abstract

We study sequential mean estimation in $\mathbb{R}^d$. In particular, we derive time-uniform confidence spheres -- confidence sphere sequences (CSSs) -- which contain the mean of random vectors with high probability simultaneously across all sample sizes. Our results include a dimension-free CSS for log-concave random vectors, a dimension-free CSS for sub-Gaussian random vectors, and CSSs for sub-$\psi$ random vectors (which includes sub-gamma, sub-Poisson, and sub-exponential distributions). Many of our results are optimal. For sub-Gaussian distributions we also provide a CSS which tracks a time-varying mean, generalizing Robbins' mixture approach to the multivariate setting. Finally, we provide several CSSs for heavy-tailed random vectors (two moments only). Our bounds hold under a martingale assumption on the mean and do not require that the observations be iid. Our work is based on PAC-Bayesian theory and inspired by an approach of Catoni and Giulini.