PAC learnability versus VC dimension: a footnote to a basic result of statistical learning

TL;DR

This paper examines the relationship between PAC learnability and VC dimension, revealing that under Martin's Axiom, PAC learnability is equivalent to having finite VC dimension even without measurability assumptions.

Contribution

It extends the fundamental PAC learnability theorem by showing the equivalence under a weaker set-theoretic assumption, Martin's Axiom, removing the need for measurability.

Findings

PAC learnability equals finite VC dimension under Martin's Axiom.

Counterexamples exist under the Continuum Hypothesis without additional assumptions.

The result clarifies the set-theoretic conditions affecting learnability and VC dimension.

Abstract

A fundamental result of statistical learnig theory states that a concept class is PAC learnable if and only if it is a uniform Glivenko-Cantelli class if and only if the VC dimension of the class is finite. However, the theorem is only valid under special assumptions of measurability of the class, in which case the PAC learnability even becomes consistent. Otherwise, there is a classical example, constructed under the Continuum Hypothesis by Dudley and Durst and further adapted by Blumer, Ehrenfeucht, Haussler, and Warmuth, of a concept class of VC dimension one which is neither uniform Glivenko-Cantelli nor consistently PAC learnable. We show that, rather surprisingly, under an additional set-theoretic hypothesis which is much milder than the Continuum Hypothesis (Martin's Axiom), PAC learnability is equivalent to finite VC dimension for every concept class.