Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound

TL;DR

This paper introduces stable sample compression schemes for supervised learning, deriving new optimal margin bounds for SVMs that improve understanding of their generalization capabilities.

Contribution

It presents a novel stable compression framework and establishes an optimal margin bound for SVM, resolving a long-standing open problem.

Findings

Derived a new SVM margin bound without the log factor

Proved the optimality of the new margin bound

Provided improved data-dependent generalization bounds for several algorithms

Abstract

We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier. We use this technique to derive a variety of novel or improved data-dependent generalization bounds for several learning algorithms. In particular, we prove a new margin bound for SVM, removing a log factor. The new bound is provably optimal. This resolves a long-standing open question about the PAC margin bounds achievable by SVM.