Sign Consistency of the Generalized Elastic Net Estimator

TL;DR

This paper introduces a new variable selection method called gEN for high-dimensional linear models with highly correlated features, using a transformed model and generalized Elastic Net, with theoretical guarantees and improved performance.

Contribution

It proposes the gEN approach with the GIC condition, extending previous irrepresentability conditions, and demonstrates its effectiveness both theoretically and through simulations.

Findings

gEN can recover true variable support under GIC.

The approach outperforms existing methods in synthetic data experiments.

Theoretical analysis confirms consistency of variable selection.

Abstract

In this paper, we propose a novel variable selection approach in the framework of high-dimensional linear models where the columns of the design matrix are highly correlated. It consists in rewriting the initial high-dimensional linear model to remove the correlation between the columns of the design matrix and in applying a generalized Elastic Net criterion since it can be seen as an extension of the generalized Lasso. The properties of our approach called gEN (generalized Elastic Net) are investigated both from a theoretical and a numerical point of view. More precisely, we provide a new condition called GIC (Generalized Irrepresentable Condition) which generalizes the EIC (Elastic Net Irrepresentable Condition) of Jia and Yu (2010) under which we prove that our estimator can recover the positions of the null and non null entries of the coefficients when the sample size tends to infinity. We also assess the performance of our methodology using synthetic data and compare it with alternative approaches. Our numerical experiments show that our approach improves the variable selection performance in many cases.