Vapnik-Chervonenkis Dimension of Axis-Parallel Cuts

TL;DR

This paper precisely calculates the VC dimension of axis-parallel half-spaces in R^d, revealing it is significantly smaller than the dimension d, approximately proportional to log_2(d), which impacts classifier performance evaluation.

Contribution

It provides an exact computation of the VC dimension for axis-parallel cuts, showing it is roughly log_2(d), a novel insight for high-dimensional classifier analysis.

Findings

VC dimension is much smaller than d

Approximate VC dimension is proportional to log_2(d)

Results aid in evaluating classifiers using axis-parallel partitions

Abstract

The Vapnik-Chervonenkis (VC) dimension of the set of half-spaces of R^d with frontiers parallel to the axes is computed exactly. It is shown that it is much smaller than the intuitive value of d. A good approximation based on the Stirling's formula proves that it is more likely of the order log\_2(d). This result may be used to evaluate the performance of classifiers or regressors based on dyadic partitioning of R^d for instance. Algorithms using axis-parallel cuts to partition R^d are often used to reduce the computational time of such estimators when d is large.