Correlated Feature Selection with Extended Exclusive Group Lasso

TL;DR

This paper introduces a fast algorithm for the exclusive group Lasso, improving feature selection in correlated biological data and addressing unknown group structures, outperforming traditional Lasso.

Contribution

The paper proposes a novel, efficient algorithm for exclusive group Lasso and a method to handle unknown group structures, enhancing feature selection in correlated high-dimensional data.

Findings

Outperforms Lasso in selecting informative features

Effective in biological datasets with correlated features

Handles unknown group structures using stability selection

Abstract

In many high dimensional classification or regression problems set in a biological context, the complete identification of the set of informative features is often as important as predictive accuracy, since this can provide mechanistic insight and conceptual understanding. Lasso and related algorithms have been widely used since their sparse solutions naturally identify a set of informative features. However, Lasso performs erratically when features are correlated. This limits the use of such algorithms in biological problems, where features such as genes often work together in pathways, leading to sets of highly correlated features. In this paper, we examine the performance of a Lasso derivative, the exclusive group Lasso, in this setting. We propose fast algorithms to solve the exclusive group Lasso, and introduce a solution to the case when the underlying group structure is unknown. The solution combines stability selection with random group allocation and introduction of artificial features. Experiments with both synthetic and real-world data highlight the advantages of this proposed methodology over Lasso in comprehensive selection of informative features.