Uniform convergence of Vapnik--Chervonenkis classes under ergodic sampling

TL;DR

This paper proves that for countable VC classes in a complete separable metric space, the relative frequencies converge uniformly under any stationary ergodic process, extending classical results without requiring mixing or regularity conditions.

Contribution

It establishes uniform convergence of relative frequencies for VC classes under ergodic sampling without additional assumptions, using a novel direct proof method.

Findings

Uniform convergence holds for countable VC classes under ergodic processes.

No mixing or regularity conditions are needed beyond ergodicity.

The result extends classical VC theory to more general stochastic processes.

Abstract

We show that if $\mathcal{X}$ is a complete separable metric space and $\mathcal{C}$ is a countable family of Borel subsets of $\mathcal{X}$ with finite VC dimension, then, for every stationary ergodic process with values in $\mathcal{X}$, the relative frequencies of sets $C\in\mathcal{C}$ converge uniformly to their limiting probabilities. Beyond ergodicity, no assumptions are imposed on the sampling process, and no regularity conditions are imposed on the elements of $\mathcal{C}$. The result extends existing work of Vapnik and Chervonenkis, among others, who have studied uniform convergence for i.i.d. and strongly mixing processes. Our method of proof is new and direct: it does not rely on symmetrization techniques, probability inequalities or mixing conditions. The uniform convergence of relative frequencies for VC-major and VC-graph classes of functions under ergodic sampling is established as a corollary of the basic result for sets.