Efficient Decision Trees for Multi-class Support Vector Machines Using Entropy and Generalization Error Estimation

TL;DR

This paper introduces efficient multi-class SVM decision tree methods that use entropy and generalization error estimation to achieve faster classification times, especially with many classes, while maintaining accuracy.

Contribution

It presents novel tree-based SVM classification methods that optimize binary classifier selection at each node for improved speed and scalability.

Findings

Faster classification times compared to traditional methods.

Comparable accuracy with large class sets.

Effective for problems requiring quick decision-making.

Abstract

We propose new methods for Support Vector Machines (SVMs) using tree architecture for multi-class classi- fication. In each node of the tree, we select an appropriate binary classifier using entropy and generalization error estimation, then group the examples into positive and negative classes based on the selected classi- fier and train a new classifier for use in the classification phase. The proposed methods can work in time complexity between O(log2N) to O(N) where N is the number of classes. We compared the performance of our proposed methods to the traditional techniques on the UCI machine learning repository using 10-fold cross-validation. The experimental results show that our proposed methods are very useful for the problems that need fast classification time or problems with a large number of classes as the proposed methods run much faster than the traditional techniques but still provide comparable accuracy.