Adaptive robust variable selection

TL;DR

This paper introduces the adaptive robust Lasso (AR-Lasso), a method for variable selection in high-dimensional, heavy-tailed data, with theoretical guarantees and practical effectiveness demonstrated through simulations.

Contribution

It develops a two-step adaptive robust Lasso procedure with oracle properties and asymptotic normality for ultra-high dimensional data with heavy tails.

Findings

AR-Lasso achieves oracle property in simulations.

Theoretical analysis confirms asymptotic normality.

Numerical studies show superior finite-sample performance.

Abstract

Heavy-tailed high-dimensional data are commonly encountered in various scientific fields and pose great challenges to modern statistical analysis. A natural procedure to address this problem is to use penalized quantile regression with weighted $L_1$-penalty, called weighted robust Lasso (WR-Lasso), in which weights are introduced to ameliorate the bias problem induced by the $L_1$-penalty. In the ultra-high dimensional setting, where the dimensionality can grow exponentially with the sample size, we investigate the model selection oracle property and establish the asymptotic normality of the WR-Lasso. We show that only mild conditions on the model error distribution are needed. Our theoretical results also reveal that adaptive choice of the weight vector is essential for the WR-Lasso to enjoy these nice asymptotic properties. To make the WR-Lasso practically feasible, we propose a two-step procedure, called adaptive robust Lasso (AR-Lasso), in which the weight vector in the second step is constructed based on the $L_1$-penalized quantile regression estimate from the first step. This two-step procedure is justified theoretically to possess the oracle property and the asymptotic normality. Numerical studies demonstrate the favorable finite-sample performance of the AR-Lasso.