Multiclass Optimal Classification Trees with SVM-splits

TL;DR

This paper introduces a new optimization-based method for constructing multiclass classification trees using SVM splits, aiming to improve classification performance through a novel hybrid approach.

Contribution

It presents a mixed integer nonlinear programming formulation for building classification trees with SVM-based splits, a novel approach in tree construction methods.

Findings

Outperforms benchmark classifiers in experiments

Effective in multiclass classification tasks

Provides a flexible optimization framework

Abstract

In this paper we present a novel mathematical optimization-based methodology to construct tree-shaped classification rules for multiclass instances. Our approach consists of building Classification Trees in which, except for the leaf nodes, the labels are temporarily left out and grouped into two classes by means of a SVM separating hyperplane. We provide a Mixed Integer Non Linear Programming formulation for the problem and report the results of an extended battery of computational experiments to assess the performance of our proposal with respect to other benchmarking classification methods.