Feature Selection in High-dimensional Spaces Using Graph-Based Methods

TL;DR

This paper introduces a nonparametric, graph-based feature selection method for high-dimensional data that effectively identifies influential features while controlling false discoveries, demonstrated through synthetic and real-world datasets.

Contribution

It presents a novel recursive graph-based algorithm for feature selection that works without distributional assumptions and outperforms existing methods.

Findings

Successfully recovers true features with high probability.

Outperforms existing methods on synthetic data.

Detects known and novel features in real datasets.

Abstract

High-dimensional feature selection is a central problem in a variety of application domains such as machine learning, image analysis, and genomics. In this paper, we propose graph-based tests as a useful basis for feature selection. We describe an algorithm for selecting informative features in high-dimensional data, where each observation comes from one of $K$ different distributions. Our algorithm can be applied in a completely nonparametric setup without any distributional assumptions on the data, and it aims at outputting those features in the data, that contribute the most to the overall distributional variation. At the heart of our method is the recursive application of distribution-free graph-based tests on subsets of the feature set, located at different depths of a hierarchical clustering tree constructed from the data. Our algorithm recovers all truly contributing features with high probability, while ensuring optimal control on false-discovery. We show the superior performance of our method over other existing ones through synthetic data, and demonstrate the utility of this method on several real-life datasets from the domains of climate change and biology, wherein our algorithm is not only able to detect known features expected to be associated with the underlying process, but also discovers novel targets that can be subsequently studied.