Enhancements of Multi-class Support Vector Machine Construction from Binary Learners using Generalization Performance

TL;DR

This paper introduces novel methods for multi-class SVM construction that leverage cross-validation-based generalization performance of binary classifiers, significantly improving accuracy and speed over traditional algorithms.

Contribution

The paper presents new algorithms (RADAG, SE, WE, VCF) that utilize cross-validation performance to enhance multi-class SVMs, outperforming existing methods.

Findings

WE outperforms Max Wins in accuracy and speed

Proposed methods significantly improve multi-class SVM accuracy

Cross-validation-based performance estimation is more effective

Abstract

We propose several novel methods for enhancing the multi-class SVMs by applying the generalization performance of binary classifiers as the core idea. This concept will be applied on the existing algorithms, i.e., the Decision Directed Acyclic Graph (DDAG), the Adaptive Directed Acyclic Graphs (ADAG), and Max Wins. Although in the previous approaches there have been many attempts to use some information such as the margin size and the number of support vectors as performance estimators for binary SVMs, they may not accurately reflect the actual performance of the binary SVMs. We show that the generalization ability evaluated via a cross-validation mechanism is more suitable to directly extract the actual performance of binary SVMs. Our methods are built around this performance measure, and each of them is crafted to overcome the weakness of the previous algorithm. The proposed methods include the Reordering Adaptive Directed Acyclic Graph (RADAG), Strong Elimination of the classifiers (SE), Weak Elimination of the classifiers (WE), and Voting based Candidate Filtering (VCF). Experimental results demonstrate that our methods give significantly higher accuracy than all of the traditional ones. Especially, WE provides significantly superior results compared to Max Wins which is recognized as the state of the art algorithm in terms of both accuracy and classification speed with two times faster in average.