MDFS - MultiDimensional Feature Selection

TL;DR

The paper introduces MDFS, an R package for multi-dimensional feature selection that considers variable interactions, demonstrating improved sensitivity and reliability over traditional one-dimensional methods using the Madelon dataset.

Contribution

It presents a novel information-theoretic algorithm and its high-performance CUDA implementation for identifying synergistic variables in feature selection.

Findings

Multidimensional analysis outperforms one-dimensional tests in sensitivity.

MDFS provides more reliable importance rankings.

Application on Madelon dataset validates effectiveness.

Abstract

Identification of informative variables in an information system is often performed using simple one-dimensional filtering procedures that discard information about interactions between variables. Such approach may result in removing some relevant variables from consideration. Here we present an R package MDFS (MultiDimensional Feature Selection) that performs identification of informative variables taking into account synergistic interactions between multiple descriptors and the decision variable. MDFS is an implementation of an algorithm based on information theory. Computational kernel of the package is implemented in C++. A high-performance version implemented in CUDA C is also available. The applications of MDFS are demonstrated using the well-known Madelon dataset that has synergistic variables by design. The dataset comes from the UCI Machine Learning Repository. It is shown that multidimensional analysis is more sensitive than one-dimensional tests and returns more reliable rankings of importance.