Dynamic Ensemble Selection Using Fuzzy Hyperboxes

TL;DR

This paper introduces FH-DES, a fuzzy hyperbox-based dynamic ensemble selection method that reduces computational complexity and is less sensitive to data distribution, while also utilizing misclassified samples for competence estimation.

Contribution

The paper proposes a novel DES framework using fuzzy hyperboxes that is computationally efficient and less sensitive to local data distribution, incorporating misclassified samples for the first time.

Findings

Achieves high classification accuracy

Lower computational complexity than state-of-the-art methods

Less sensitive to local data distribution

Abstract

Most dynamic ensemble selection (DES) methods utilize the K-Nearest Neighbors (KNN) algorithm to estimate the competence of classifiers in a small region surrounding the query sample. However, KNN is very sensitive to the local distribution of the data. Moreover, it also has a high computational cost as it requires storing the whole data in memory and performing multiple distance calculations during inference. Hence, the dependency on the KNN algorithm ends up limiting the use of DES techniques for large-scale problems. This paper presents a new DES framework based on fuzzy hyperboxes called FH-DES. Each hyperbox can represent a group of samples using only two data points (Min and Max corners). Thus, the hyperbox-based system will have less computational complexity than other dynamic selection methods. In addition, despite the KNN-based approaches, the fuzzy hyperbox is not sensitive to the local data distribution. Therefore, the local distribution of the samples does not affect the system's performance. Furthermore, in this research, for the first time, misclassified samples are used to estimate the competence of the classifiers, which has not been observed in previous fusion approaches. Experimental results demonstrate that the proposed method has high classification accuracy while having a lower complexity when compared with the state-of-the-art dynamic selection methods. The implemented code is available at https://github.com/redavtalab/FH-DES_IJCNN.git.