Post-Lasso Inference for High-Dimensional Regression

TL;DR

This paper introduces a novel variable selection method for high-dimensional regression that improves accuracy by controlling noise variable inclusion using post-selection inference techniques.

Contribution

It proposes a new approach to variable selection that identifies the positions of relevant and noise variables on the Lasso path, enhancing selection precision.

Findings

Method outperforms existing techniques in selection accuracy.

Provides better interpretability of selected variables.

Utilizes covariance and Q statistics for post-selection inference.

Abstract

Among the most popular variable selection procedures in high-dimensional regression, Lasso provides a solution path to rank the variables and determines a cut-off position on the path to select variables and estimate coefficients. In this paper, we consider variable selection from a new perspective motivated by the frequently occurred phenomenon that relevant variables are not completely distinguishable from noise variables on the solution path. We propose to characterize the positions of the first noise variable and the last relevant variable on the path. We then develop a new variable selection procedure to control over-selection of the noise variables ranking after the last relevant variable, and, at the same time, retain a high proportion of relevant variables ranking before the first noise variable. Our procedure utilizes the recently developed covariance test statistic and Q statistic in post-selection inference. In numerical examples, our method compares favorably with other existing methods in selection accuracy and the ability to interpret its results.