Faster Support Vector Machines

TL;DR

This paper introduces a faster multilevel support vector machine approach that constructs problem hierarchies using label propagation, significantly reducing training time while maintaining classification accuracy.

Contribution

It presents a novel multilevel SVM training method utilizing label propagation for hierarchy construction, achieving up to orders of magnitude faster training.

Findings

Up to 15 times faster training than ThunderSVM.

Comparable classification quality to existing methods.

Effective hierarchy construction with label propagation.

Abstract

The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- optimization step, multilevel SVMs first build a hierarchy of problems decreasing in size that resemble the original problem and then train an SVM model for each hierarchy level, benefiting from the solved models of previous levels. We present a faster multilevel support vector machine that uses a label propagation algorithm to construct the problem hierarchy. Extensive experiments indicate that our approach is up to orders of magnitude faster than the previous fastest algorithm while having comparable classification quality. For example, already one of our sequential solvers is on average a factor 15 faster than the parallel ThunderSVM algorithm, while having similar classification quality.