Tight Bounds for Local Glivenko-Cantelli

TL;DR

This paper derives tight, non-asymptotic bounds for the maximum deviation between empirical and true means in high-dimensional Bernoulli distributions, addressing an open problem and extending classical results.

Contribution

It provides the exact non-asymptotic behavior of the maximum deviation for product distributions, solving a COLT 2023 open problem and establishing the tightest bounds based on mean statistics.

Findings

Exact non-asymptotic bounds for product distributions.

Demonstration of the necessity of a specific factor in upper bounds.

Extension of bounds to general distributions and other norms.

Abstract

This paper addresses the statistical problem of estimating the infinite-norm deviation from the empirical mean to the distribution mean for high-dimensional distributions on $\{0,1\}^d$, potentially with $d=\infty$. Unlike traditional bounds as in the classical Glivenko-Cantelli theorem, we explore the instance-dependent convergence behavior. For product distributions, we provide the exact non-asymptotic behavior of the expected maximum deviation, revealing various regimes of decay. In particular, these tight bounds demonstrate the necessity of a previously proposed factor for an upper bound, answering a corresponding COLT 2023 open problem. We also consider general distributions on $\{0,1\}^d$ and provide the tightest possible bounds for the maximum deviation of the empirical mean given only the mean statistic. Along the way, we prove a localized version of the Dvoretzky-Kiefer-Wolfowitz inequality. Additionally, we present some results for two other cases, one where the deviation is measured in some $q$-norm, and the other where the distribution is supported on a continuous domain $[0,1]^d$, and also provide some high-probability bounds for the maximum deviation in the independent Bernoulli case.