L p -norm Sauer-Shelah Lemma for Margin Multi-category Classifiers

TL;DR

This paper introduces an L p-norm Sauer-Shelah lemma for margin multi-category classifiers, providing improved risk bounds with sublinear dependency on the number of categories, advancing the theoretical understanding of classifier complexity.

Contribution

The paper establishes a new L p-norm Sauer-Shelah lemma for margin classifiers, improving risk bounds and clarifying complexity dependence on the number of categories.

Findings

Derived risk bounds in L ∞ and L 2 norms with sublinear C dependency.

Established an L p-norm Sauer-Shelah lemma for vector-valued function classes.

Improved theoretical bounds over previous results.

Abstract

In the framework of agnostic learning, one of the main open problems of the theory of multi-category pattern classification is the characterization of the way the complexity varies with the number C of categories. More precisely, if the classifier is characterized only through minimal learnability hypotheses, then the optimal dependency on C that an upper bound on the probability of error should exhibit is unknown. We consider margin classifiers. They are based on classes of vector-valued functions with one component function per category, and the classes of component functions are uniform Glivenko-Cantelli classes. For these classifiers, an L p-norm Sauer-Shelah lemma is established. It is then used to derive guaranteed risks in the L $\infty$ and L 2-norms. These bounds improve over the state-of-the-art ones with respect to their dependency on C, which is sublinear.