IMMIGRATE: A Margin-based Feature Selection Method with Interaction Terms

TL;DR

IMMIGRATE is a novel feature selection algorithm that effectively captures feature interactions by incorporating interaction terms and margin principles, demonstrating robustness and superior performance across multiple tasks.

Contribution

This paper introduces IMMIGRATE, a new feature selection method that explicitly models interaction terms and combines local and global information for improved robustness.

Findings

Achieves state-of-the-art results on several tasks

Demonstrates robustness and compatibility with Boosting

Effectively differentiates interaction effects from marginal effects

Abstract

Relief based algorithms have often been claimed to uncover feature interactions. However, it is still unclear whether and how interaction terms will be differentiated from marginal effects. In this paper, we propose IMMIGRATE algorithm by including and training weights for interaction terms. Besides applying the large margin principle, we focus on the robustness of the contributors of margin and consider local and global information simultaneously. Moreover, IMMIGRATE has been shown to enjoy attractive properties, such as robustness and combination with Boosting. We evaluate our proposed method on several tasks, which achieves state-of-the-art results significantly.