Enhancing Diversity in Multi-objective Feature Selection

TL;DR

This paper improves multi-objective feature selection by enhancing population diversity in NSGA-II through new initialization and re-initialization methods, leading to better performance on real-world classification tasks.

Contribution

It introduces a novel diversity augmentation technique for NSGA-II, significantly improving feature selection quality in high-dimensional problems.

Findings

Enhanced population diversity improves algorithm performance.

Re-initialization with random individuals boosts feature selection quality.

Method tested successfully on twelve real-world datasets.

Abstract

Feature selection plays a pivotal role in the data preprocessing and model-building pipeline, significantly enhancing model performance, interpretability, and resource efficiency across diverse domains. In population-based optimization methods, the generation of diverse individuals holds utmost importance for adequately exploring the problem landscape, particularly in highly multi-modal multi-objective optimization problems. Our study reveals that, in line with findings from several prior research papers, commonly employed crossover and mutation operations lack the capability to generate high-quality diverse individuals and tend to become confined to limited areas around various local optima. This paper introduces an augmentation to the diversity of the population in the well-established multi-objective scheme of the genetic algorithm, NSGA-II. This enhancement is achieved through two key components: the genuine initialization method and the substitution of the worst individuals with new randomly generated individuals as a re-initialization approach in each generation. The proposed multi-objective feature selection method undergoes testing on twelve real-world classification problems, with the number of features ranging from 2,400 to nearly 50,000. The results demonstrate that replacing the last front of the population with an equivalent number of new random individuals generated using the genuine initialization method and featuring a limited number of features substantially improves the population's quality and, consequently, enhances the performance of the multi-objective algorithm.