DCSVM: Fast Multi-class Classification using Support Vector Machines

TL;DR

DCSVM is a divide and conquer algorithm that efficiently performs multi-class classification with SVMs by recursively partitioning data, reducing classes at each step, and achieving logarithmic decision complexity in the best case.

Contribution

The paper introduces DCSVM, a novel divide and conquer approach that improves multi-class SVM classification efficiency through smart data partitioning and recursive class elimination.

Findings

Achieves $O( ext{log }k)$ decision steps in best case

Reduces the number of candidate classes efficiently at each step

Performs comparably to existing techniques in worst case

Abstract

We present DCSVM, an efficient algorithm for multi-class classification using Support Vector Machines. DCSVM is a divide and conquer algorithm which relies on data sparsity in high dimensional space and performs a smart partitioning of the whole training data set into disjoint subsets that are easily separable. A single prediction performed between two partitions eliminates at once one or more classes in one partition, leaving only a reduced number of candidate classes for subsequent steps. The algorithm continues recursively, reducing the number of classes at each step, until a final binary decision is made between the last two classes left in the competition. In the best case scenario, our algorithm makes a final decision between $k$ classes in $O(\log k)$ decision steps and in the worst case scenario DCSVM makes a final decision in $k-1$ steps, which is not worse than the existent techniques.