Uniform Convergence Beyond Glivenko-Cantelli

TL;DR

This paper extends the concept of uniform convergence beyond the classical empirical mean estimator, introducing UME-learnability to characterize when collections of distributions allow for uniform mean estimation by any estimator, and proves properties of such collections.

Contribution

It introduces UME-learnability as a new framework for uniform mean estimation beyond empirical means and characterizes its conditions, including separability and closure properties.

Findings

Separability of mean vectors is sufficient for UME-learnability.

Non-separable mean vectors can still be UME-learnable with different techniques.

Countable unions of UME-learnable collections are UME-learnable.

Abstract

We characterize conditions under which collections of distributions on $\{0,1\}^\mathbb{N}$ admit uniform estimation of their mean. Prior work from Vapnik and Chervonenkis (1971) has focused on uniform convergence using the empirical mean estimator, leading to the principle known as $P-$ Glivenko-Cantelli. We extend this framework by moving beyond the empirical mean estimator and introducing Uniform Mean Estimability, also called UME-learnability, which captures when a collection permits uniform mean estimation by any arbitrary estimator. We work on the space created by the mean vectors of the collection of distributions. For each distribution, the mean vector records the expected value in each coordinate. We show that separability of the mean vectors is a sufficient condition for UME-learnability. However, we show that separability of the mean vectors is not necessary for UME-learnability by constructing a collection of distributions whose mean vectors are non-separable yet UME-learnable using techniques fundamentally different from those used in our separability-based analysis. Finally, we establish that countable unions of UME-learnable collections are also UME-learnable, solving the conjecture posed in Cohen et al. (2025).