Error Correction in Learning using SVMs

TL;DR

This paper introduces SubSVMs, a practical error-correction method for learning binary classifiers under adversarial label-noise, leveraging class-balanced subsampling and bagging to recover clean data and improve robustness.

Contribution

The paper proposes SubSVMs, a novel error-correction algorithm that uses small, class-balanced subsets and majority voting to handle label noise in SVM learning.

Findings

Effective noise-tolerance demonstrated on synthetic and benchmark data.

Significant run-time improvements over standard SVMs.

Highlights importance of class-balanced sampling and subsampled bagging.

Abstract

This paper is concerned with learning binary classifiers under adversarial label-noise. We introduce the problem of error-correction in learning where the goal is to recover the original clean data from a label-manipulated version of it, given (i) no constraints on the adversary other than an upper-bound on the number of errors, and (ii) some regularity properties for the original data. We present a simple and practical error-correction algorithm called SubSVMs that learns individual SVMs on several small-size (log-size), class-balanced, random subsets of the data and then reclassifies the training points using a majority vote. Our analysis reveals the need for the two main ingredients of SubSVMs, namely class-balanced sampling and subsampled bagging. Experimental results on synthetic as well as benchmark UCI data demonstrate the effectiveness of our approach. In addition to noise-tolerance, log-size subsampled bagging also yields significant run-time benefits over standard SVMs.