Dynamic Ensemble Selection VS K-NN: why and when Dynamic Selection obtains higher classification performance?

TL;DR

This paper compares dynamic selection (DS) techniques with K-NN classifiers, demonstrating that DS methods outperform K-NN especially on hard-to-classify samples, and explains when and why DS should be used.

Contribution

The paper provides an extensive analysis of DS techniques versus K-NN, highlighting the conditions under which DS achieves higher accuracy and explaining the reasons behind this performance boost.

Findings

DS methods outperform K-NN on 30 datasets.

DS techniques handle high-instance hardness better.

DS outperforms K-NN especially near decision boundaries.

Abstract

Multiple classifier systems focus on the combination of classifiers to obtain better performance than a single robust one. These systems unfold three major phases: pool generation, selection and integration. One of the most promising MCS approaches is Dynamic Selection (DS), which relies on finding the most competent classifier or ensemble of classifiers to predict each test sample. The majority of the DS techniques are based on the K-Nearest Neighbors (K-NN) definition, and the quality of the neighborhood has a huge impact on the performance of DS methods. In this paper, we perform an analysis comparing the classification results of DS techniques and the K-NN classifier under different conditions. Experiments are performed on 18 state-of-the-art DS techniques over 30 classification datasets and results show that DS methods present a significant boost in classification accuracy even though they use the same neighborhood as the K-NN. The reasons behind the outperformance of DS techniques over the K-NN classifier reside in the fact that DS techniques can deal with samples with a high degree of instance hardness (samples that are located close to the decision border) as opposed to the K-NN. In this paper, not only we explain why DS techniques achieve higher classification performance than the K-NN but also when DS should be used.