Generating Redundant Features with Unsupervised Multi-Tree Genetic Programming

TL;DR

This paper introduces a novel genetic programming method to automatically generate complex, redundant features for high-dimensional datasets, aiding in the evaluation of feature selection algorithms.

Contribution

The work presents one of the first methods for automatically creating complex redundant features using multi-tree genetic programming.

Findings

Generated features are difficult and redundant, suitable for benchmarking.

Method successfully creates synthetic features that challenge feature selection algorithms.

Potential to improve evaluation of feature selection methods.

Abstract

Recently, feature selection has become an increasingly important area of research due to the surge in high-dimensional datasets in all areas of modern life. A plethora of feature selection algorithms have been proposed, but it is difficult to truly analyse the quality of a given algorithm. Ideally, an algorithm would be evaluated by measuring how well it removes known bad features. Acquiring datasets with such features is inherently difficult, and so a common technique is to add synthetic bad features to an existing dataset. While adding noisy features is an easy task, it is very difficult to automatically add complex, redundant features. This work proposes one of the first approaches to generating redundant features, using a novel genetic programming approach. Initial experiments show that our proposed method can automatically create difficult, redundant features which have the potential to be used for creating high-quality feature selection benchmark datasets.