An Evolutionary Correlation-aware Feature Selection Method for Classification Problems

TL;DR

This paper introduces a novel evolutionary feature selection method that efficiently considers feature interactions and correlations, improving classification accuracy while reducing computational time.

Contribution

It proposes an estimation of distribution algorithm that dynamically determines feature subset size and models feature interactions, enhancing selection quality and convergence speed.

Findings

Outperforms state-of-the-art methods on UCI datasets

Effectively detects correlated and complementary features

Reduces time complexity compared to traditional algorithms

Abstract

The population-based optimization algorithms have provided promising results in feature selection problems. However, the main challenges are high time complexity. Moreover, the interaction between features is another big challenge in FS problems that directly affects the classification performance. In this paper, an estimation of distribution algorithm is proposed to meet three goals. Firstly, as an extension of EDA, the proposed method generates only two individuals in each iteration that compete based on a fitness function and evolve during the algorithm, based on our proposed update procedure. Secondly, we provide a guiding technique for determining the number of features for individuals in each iteration. As a result, the number of selected features of the final solution will be optimized during the evolution process. The two mentioned advantages can increase the convergence speed of the algorithm. Thirdly, as the main contribution of the paper, in addition to considering the importance of each feature alone, the proposed method can consider the interaction between features. Thus, it can deal with complementary features and consequently increase classification performance. To do this, we provide a conditional probability scheme that considers the joint probability distribution of selecting two features. The introduced probabilities successfully detect correlated features. Experimental results on a synthetic dataset with correlated features prove the performance of our proposed approach facing these types of features. Furthermore, the results on 13 real-world datasets obtained from the UCI repository show the superiority of the proposed method in comparison with some state-of-the-art approaches.