A learning problem whose consistency is equivalent to the non-existence of real-valued measurable cardinals

TL;DR

This paper links the universal consistency of the k-nearest neighbor learning rule in metric spaces to the set-theoretic concept of real-valued measurable cardinals, showing the consistency depends on their existence.

Contribution

It establishes a novel connection between statistical learning theory and set theory, specifically relating the consistency of k-NN classifiers to the existence of real-valued measurable cardinals.

Findings

k-NN is universally consistent iff in every separable subspace and density is below real-measurable cardinals

Universal consistency in certain metric spaces depends on the non-existence of real-valued measurable cardinals

Results are consistent with ZFC but cannot be proved within ZFC alone.

Abstract

We show that the $k$-nearest neighbour learning rule is universally consistent in a metric space $X$ if and only if it is universally consistent in every separable subspace of $X$ and the density of $X$ is less than every real-measurable cardinal. In particular, the $k$-NN classifier is universally consistent in every metric space whose separable subspaces are sigma-finite dimensional in the sense of Nagata and Preiss if and only if there are no real-valued measurable cardinals. The latter assumption is relatively consistent with ZFC, however the consistency of the existence of such cardinals cannot be proved within ZFC. Our results were inspired by an example sketched by C\'erou and Guyader in 2006 at an intuitive level of rigour.