IIFE: Interaction Information Based Automated Feature Engineering

TL;DR

This paper introduces IIFE, an automated feature engineering algorithm based on interaction information, which improves feature selection by identifying synergistic feature pairs, demonstrating superior performance over existing methods.

Contribution

The paper presents a novel AutoFE algorithm called IIFE that leverages interaction information to enhance feature engineering and also improves existing AutoFE algorithms.

Findings

IIFE outperforms existing AutoFE algorithms in predictive tasks.

Interaction information effectively identifies synergistic feature pairs.

Proper experimental setup is crucial for evaluating AutoFE methods.

Abstract

Automated feature engineering (AutoFE) is the process of automatically building and selecting new features that help improve downstream predictive performance. While traditional feature engineering requires significant domain expertise and time-consuming iterative testing, AutoFE strives to make feature engineering easy and accessible to all data science practitioners. We introduce a new AutoFE algorithm, IIFE, based on determining which feature pairs synergize well through an information-theoretic perspective called interaction information. We demonstrate the superior performance of IIFE over existing algorithms. We also show how interaction information can be used to improve existing AutoFE algorithms. Finally, we highlight several critical experimental setup issues in the existing AutoFE literature and their effects on performance.