Building an Ensemble of Classifiers via Randomized Models of Ensemble Members

TL;DR

This paper introduces a new randomized model for ensemble classifiers based on random selection of learning sets, providing a probabilistic measure of classifier competence and demonstrating superior performance on benchmark datasets.

Contribution

The paper develops a novel randomized classifier model using random learning set selection, with mathematical foundations and practical competence estimation for ensemble systems.

Findings

Achieved lowest ranks in most quality criteria across 67 datasets.

Demonstrated improved ensemble performance over previous randomized models.

Validated the approach with extensive experimental evaluation.

Abstract

Many dynamic ensemble selection (DES) methods are known in the literature. A previously-developed by the authors, method consists in building a randomized classifier which is treated as a model of the base classifier. The model is equivalent to the base classifier in a certain probabilistic sense. Next, the probability of correct classification of randomized classifier is taken as the competence of the evaluated classifier. In this paper, a novel randomized model of base classifier is developed. In the proposed method, the random operation of the model results from a random selection of the learning set from the family of learning sets of a fixed size. The paper presents the mathematical foundations of this approach and shows how, for a practical application when learning and validation sets are given, one can determine the measure of competence and build a MC system with the DES scheme. The DES scheme with the proposed model of competence was experimentally evaluated on the collection of 67 benchmark datasets and compared in terms of eight quality criteria with two ensemble classifiers which use the previously-proposed concepts of randomized model. The proposed approach achieved the lowest ranks for almost all investigated quality criteria.