Combinatorial and Structural Results for gamma-Psi-dimensions

TL;DR

This paper introduces gamma-Psi-dimensions for analyzing the generalization performance of multi-category classifiers, providing new bounds that improve upon existing measures and facilitate multi-class to binary transition analysis.

Contribution

It develops combinatorial and structural results for gamma-Psi-dimensions, enhancing capacity measure bounds and offering a promising alternative to traditional complexity measures.

Findings

Gamma-Psi-dimensions improve bounds over fat-shattering dimensions.

They enable better handling of classifier features in capacity estimates.

Results suggest gamma-Psi-dimensions are effective for multi-class to binary transition.

Abstract

This article deals with the generalization performance of margin multi-category classifiers, when minimal learnability hypotheses are made. In that context, the derivation of a guaranteed risk is based on the handling of capacity measures belonging to three main families: Rademacher/Gaussian complexities, metric entropies and scale-sensitive combinatorial dimensions. The scale-sensitive combinatorial dimensions dedicated to the classifiers of this kind are the gamma-Psi-dimensions. We introduce the combinatorial and structural results needed to involve them in the derivation of guaranteed risks and establish the corresponding upper bounds on the metric entropies and the Rademacher complexity. Two major conclusions can be drawn: 1. the gamma-Psi-dimensions always bring an improvement compared to the use of the fat-shattering dimension of the class of margin functions; 2. thanks to their capacity to take into account basic features of the classifier, they represent a promising alternative to performing the transition from the multi-class case to the binary one with covering numbers.