Distributed Optimization of Multi-Class SVMs

TL;DR

This paper develops distributed algorithms for all-in-one multi-class SVMs, enabling scalable training and comparison with one-vs-rest SVMs, demonstrating superior accuracy on text classification tasks.

Contribution

It introduces distributed algorithms for all-in-one SVM formulations, allowing efficient training on large-scale multi-class data and enabling direct comparison with one-vs-rest SVMs.

Findings

Distributed algorithms enable scalable training of all-in-one SVMs.

All-in-one SVMs show superior accuracy on text classification.

Comparison with one-vs-rest SVMs highlights advantages of all-in-one models.

Abstract

Training of one-vs.-rest SVMs can be parallelized over the number of classes in a straight forward way. Given enough computational resources, one-vs.-rest SVMs can thus be trained on data involving a large number of classes. The same cannot be stated, however, for the so-called all-in-one SVMs, which require solving a quadratic program of size quadratically in the number of classes. We develop distributed algorithms for two all-in-one SVM formulations (Lee et al. and Weston and Watkins) that parallelize the computation evenly over the number of classes. This allows us to compare these models to one-vs.-rest SVMs on unprecedented scale. The results indicate superior accuracy on text classification data.