Semi-standard partial covariance variable selection when irrepresentable conditions fail

TL;DR

This paper introduces the semi-standard partial covariance (SPAC) method, which improves variable selection in high-dimensional data when irrepresentable conditions fail, by effectively reducing correlation effects and capturing true predictor effects.

Contribution

The paper proposes the SPAC approach with Lasso and SCAD penalties, demonstrating strong sign consistency and superior performance over existing methods under irrepresentable condition violations.

Findings

SPAC achieves sign consistency in high-dimensional settings.

Numerical studies show SPAC outperforms existing methods.

Application to PTSD data confirms practical effectiveness.

Abstract

Traditional variable selection methods could fail to be sign consistent when irrepresentable conditions are violated. This is especially critical in high-dimensional settings when the number of predictors exceeds the sample size. In this paper, we propose a new semi-standard partial covariance (SPAC) approach which is capable of reducing correlation effects from other covariates while fully capturing the magnitude of coefficients. The proposed SPAC is effective in choosing covariates which have direct effects on the response variable, while eliminating the predictors which are not directly associated with the response but are highly correlated with the relevant predictors. We show that the proposed SPAC method with the Lasso penalty or the smoothly clipped absolute deviation (SCAD) penalty possesses strong sign consistency in high-dimensional settings. Numerical studies and a post-traumatic stress disorder data application also confirm that the proposed method outperforms the existing Lasso, adaptive Lasso, SCAD, Peter-Clark-simple algorithm, and factor-adjusted regularized model selection methods when the irrepresentable conditions fail.