Enhancing Multi-Class Classification of Random Forest using Random Vector Functional Neural Network and Oblique Decision Surfaces

TL;DR

This paper introduces a novel ensemble classification method combining a fast neural network with oblique decision trees, improving multi-class classification performance by effectively handling hard-to-classify samples.

Contribution

It presents a new oblique decision tree variant and a fusion ensemble method that leverages a rapid neural network for enhanced multi-class classification.

Findings

Outperforms state-of-the-art classifiers on multiple datasets

Efficient training due to closed-form neural network solution

Effectively groups confusing samples for fine-grained classification

Abstract

Both neural networks and decision trees are popular machine learning methods and are widely used to solve problems from diverse domains. These two classifiers are commonly used base classifiers in an ensemble framework. In this paper, we first present a new variant of oblique decision tree based on a linear classifier, then construct an ensemble classifier based on the fusion of a fast neural network, random vector functional link network and oblique decision trees. Random Vector Functional Link Network has an elegant closed form solution with extremely short training time. The neural network partitions each training bag (obtained using bagging) at the root level into C subsets where C is the number of classes in the dataset and subsequently, C oblique decision trees are trained on such partitions. The proposed method provides a rich insight into the data by grouping the confusing or hard to classify samples for each class and thus, provides an opportunity to employ fine-grained classification rule over the data. The performance of the ensemble classifier is evaluated on several multi-class datasets where it demonstrates a superior performance compared to other state-of- the-art classifiers.