A Refinement of Vapnik--Chervonenkis' Theorem

TL;DR

This paper refines Vapnik--Chervonenkis' theorem by replacing Hoeffding's inequality with a normal approximation, resulting in a sharper estimate for the rate of uniform convergence of empirical to theoretical probabilities.

Contribution

It introduces a normal approximation approach with Berry--Esseen bounds to improve the classical VC theorem's convergence rate estimates.

Findings

Achieves a moderate-deviation sharpening of the VC estimate

Provides explicit error control via Berry--Esseen bounds

Introduces a new probabilistic technique for VC analysis

Abstract

Vapnik--Chervonenkis' theorem is a seminal result in machine learning. It establishes sufficient conditions for empirical probabilities to converge to theoretical probabilities, uniformly over families of events. It also provides an estimate for the rate of such uniform convergence. We revisit the probabilistic component of the classical argument. Instead of applying Hoeffding's inequality at the final step, we use a normal approximation with explicit Berry--Esseen error control. This yields a moderate-deviation sharpening of the usual VC estimate, with an additional factor of order $(\varepsilon\sqrt{n})^{-1}$ in the leading exponential term when $\varepsilon\sqrt{n}$ is large.