Engineering fast multilevel support vector machines

TL;DR

This paper introduces a multilevel framework for support vector machines that significantly speeds up training on large-scale and imbalanced datasets while maintaining high classification quality, using PETSc for integration.

Contribution

The authors develop a generalized multilevel framework for regular and weighted SVMs that balances classification quality and computational efficiency, with implementation details and reproducibility resources.

Findings

Significant speed-up over existing nonlinear SVM libraries.

Effective handling of large-scale and imbalanced datasets.

Framework easily integrates with scientific computing tasks.

Abstract

The computational complexity of solving nonlinear support vector machine (SVM) is prohibitive on large-scale data. In particular, this issue becomes very sensitive when the data represents additional difficulties such as highly imbalanced class sizes. Typically, nonlinear kernels produce significantly higher classification quality to linear kernels but introduce extra kernel and model parameters which requires computationally expensive fitting. This increases the quality but also reduces the performance dramatically. We introduce a generalized fast multilevel framework for regular and weighted SVM and discuss several versions of its algorithmic components that lead to a good trade-off between quality and time. Our framework is implemented using PETSc which allows an easy integration with scientific computing tasks. The experimental results demonstrate significant speed up compared to the state-of-the-art nonlinear SVM libraries. Reproducibility: our source code, documentation and parameters are available at https:// github.com/esadr/mlsvm.