A High-dimensional M-estimator Framework for Bi-level Variable Selection

TL;DR

This paper introduces a high-dimensional M-estimator framework for bi-level variable selection that is robust to heavy-tailed data and outliers, using a two-stage procedure with theoretical guarantees and demonstrated effectiveness.

Contribution

It proposes a novel two-stage penalized M-estimator framework for bi-level variable selection with theoretical consistency results in high-dimensional settings.

Findings

The method achieves local estimation and selection consistency.

Simulation studies show strong finite sample performance.

Real data analysis confirms practical effectiveness.

Abstract

In high-dimensional data analysis, bi-level sparsity is often assumed when covariates function group-wisely and sparsity can appear either at the group level or within certain groups. In such cases, an ideal model should be able to encourage the bi-level variable selection consistently. Bi-level variable selection has become even more challenging when data have heavy-tailed distribution or outliers exist in random errors and covariates. In this paper, we study a framework of high-dimensional M-estimation for bi-level variable selection. This framework encourages bi-level sparsity through a computationally efficient two-stage procedure. In theory, we provide sufficient conditions under which our two-stage penalized M-estimator possesses simultaneous local estimation consistency and the bi-level variable selection consistency if certain nonconvex penalty functions are used at the group level. Both our simulation studies and real data analysis demonstrate satisfactory finite sample performance of the proposed estimators under different irregular settings.