Discovering Support and Affiliated Features from Very High Dimensions

TL;DR

This paper introduces a new high-dimensional feature selection method that identifies both key support features and their correlated affiliated features, improving prediction accuracy and interpretability.

Contribution

It proposes an efficient embedded feature selection algorithm that simultaneously discovers support features and their affiliated groups using correlation constraints.

Findings

Significant prediction performance improvements over existing methods.

Effective identification of support and affiliated feature groups.

Validated on synthetic and real-world high-dimensional datasets.

Abstract

In this paper, a novel learning paradigm is presented to automatically identify groups of informative and correlated features from very high dimensions. Specifically, we explicitly incorporate correlation measures as constraints and then propose an efficient embedded feature selection method using recently developed cutting plane strategy. The benefits of the proposed algorithm are two-folds. First, it can identify the optimal discriminative and uncorrelated feature subset to the output labels, denoted here as Support Features, which brings about significant improvements in prediction performance over other state of the art feature selection methods considered in the paper. Second, during the learning process, the underlying group structures of correlated features associated with each support feature, denoted as Affiliated Features, can also be discovered without any additional cost. These affiliated features serve to improve the interpretations on the learning tasks. Extensive empirical studies on both synthetic and very high dimensional real-world datasets verify the validity and efficiency of the proposed method.