Rank-based Lasso -- efficient methods for high-dimensional robust model selection

TL;DR

This paper introduces RankLasso, a robust and efficient method for high-dimensional model selection that uses rank-based responses and extends to thresholded and adaptive versions, outperforming existing methods like LADLasso.

Contribution

The paper proposes new consistency results for RankLasso and its variants, broadening its applicability in diverse data scenarios with heavy-tailed errors and nonlinear link functions.

Findings

RankLasso effectively identifies relevant predictors with Cauchy noise.

Modified RankLasso variants outperform the original in correlated predictor settings.

RankLasso surpasses LADLasso in model selection accuracy.

Abstract

We consider the problem of identifying significant predictors in large data bases, where the response variable depends on the linear combination of explanatory variables through an unknown link function, corrupted with the noise from the unknown distribution. We utilize the natural, robust and efficient approach, which relies on replacing values of the response variables by their ranks and then identifying significant predictors by using well known Lasso. We provide new consistency results for the proposed procedure (called ,,RankLasso") and extend the scope of its applications by proposing its thresholded and adaptive versions. Our theoretical results show that these modifications can identify the set of relevant predictors under much wider range of data generating scenarios than regular RankLasso. Theoretical results are supported by the simulation study and the real data analysis, which show that our methods can properly identify relevant predictors, even when the error terms come from the Cauchy distribution and the link function is nonlinear. They also demonstrate the superiority of the modified versions of RankLasso in the case when predictors are substantially correlated. The numerical study shows also that RankLasso performs substantially better in model selection than LADLasso, which is a well established methodology for robust model selection.