Data structure > labels? Unsupervised heuristics for SVM hyperparameter estimation

TL;DR

This paper introduces improved unsupervised heuristics for SVM hyperparameter estimation that outperform existing heuristics and are statistically comparable to the traditional grid search cross-validation method, especially when labeled data is scarce.

Contribution

The paper proposes new heuristics for SVM parameter selection that are faster and more effective than existing methods, challenging the reliance on label-dependent grid search.

Findings

Improved heuristics outperform state-of-the-art unsupervised methods.

Heuristics are statistically comparable to grid search cross-validation.

Method is effective on over 30 standard datasets.

Abstract

Classification is one of the main areas of pattern recognition research, and within it, Support Vector Machine (SVM) is one of the most popular methods outside of field of deep learning -- and a de-facto reference for many Machine Learning approaches. Its performance is determined by parameter selection, which is usually achieved by a time-consuming grid search cross-validation procedure (GSCV). That method, however relies on the availability and quality of labelled examples and thus, when those are limited can be hindered. To address that problem, there exist several unsupervised heuristics that take advantage of the characteristics of the dataset for selecting parameters instead of using class label information. While an order of magnitude faster, they are scarcely used under the assumption that their results are significantly worse than those of grid search. To challenge that assumption, we have proposed improved heuristics for SVM parameter selection and tested it against GSCV and state of the art heuristics on over 30 standard classification datasets. The results show not only its advantage over state-of-art heuristics but also that it is statistically no worse than GSCV.