AML-SVM: Adaptive Multilevel Learning with Support Vector Machines

TL;DR

This paper introduces AML-SVM, an adaptive multilevel framework for nonlinear SVMs that enhances classification accuracy and significantly reduces training time by leveraging hierarchical learning and parallel processing.

Contribution

It presents a novel multilevel learning approach for nonlinear SVMs that integrates parameter fitting and adaptive stopping to improve efficiency and performance.

Findings

Reduced variance in predictions across hierarchy levels

Significant speedup over state-of-the-art nonlinear SVMs

Maintained classification quality despite faster training

Abstract

The support vector machines (SVM) is one of the most widely used and practical optimization based classification models in machine learning because of its interpretability and flexibility to produce high quality results. However, the big data imposes a certain difficulty to the most sophisticated but relatively slow versions of SVM, namely, the nonlinear SVM. The complexity of nonlinear SVM solvers and the number of elements in the kernel matrix quadratically increases with the number of samples in training data. Therefore, both runtime and memory requirements are negatively affected. Moreover, the parameter fitting has extra kernel parameters to tune, which exacerbate the runtime even further. This paper proposes an adaptive multilevel learning framework for the nonlinear SVM, which addresses these challenges, improves the classification quality across the refinement process, and leverages multi-threaded parallel processing for better performance. The integration of parameter fitting in the hierarchical learning framework and adaptive process to stop unnecessary computation significantly reduce the running time while increase the overall performance. The experimental results demonstrate reduced variance on prediction over validation and test data across levels in the hierarchy, and significant speedup compared to state-of-the-art nonlinear SVM libraries without a decrease in the classification quality. The code is accessible at https://github.com/esadr/amlsvm.