Air-HOLP: Adaptive Regularized Feature Screening for High Dimensional Correlated Data

TL;DR

Air-HOLP introduces an adaptive, data-driven regularized feature screening method for high-dimensional correlated data, improving feature selection stability and efficiency over existing techniques.

Contribution

It advances HOLP feature screening by developing Air-HOLP, which adaptively tunes regularization parameters iteratively for better high-dimensional feature screening.

Findings

Air-HOLP outperforms traditional methods in simulations.

The method is computationally efficient and stable.

Empirical tests on genetic data validate its effectiveness.

Abstract

Handling high-dimensional datasets presents substantial computational challenges, particularly when the number of features far exceeds the number of observations and when features are highly correlated. A modern approach to mitigate these issues is feature screening. In this work, the High-dimensional Ordinary Least-squares Projection (HOLP) feature screening method is advanced by employing adaptive ridge regularization. The impact of the ridge tuning parameter on the Ridge-HOLP method is examined and Adaptive iterative ridge-HOLP (Air-HOLP) is proposed, a data-adaptive advance to Ridge-HOLP where the ridge-regularization tuning parameter is selected iteratively and optimally for better feature screening performance. The proposed method addresses the challenges of tuning parameter selection in high dimensions by offering a computationally efficient and stable alternative to traditional methods like bootstrapping and cross-validation. Air-HOLP is evaluated using simulated data and a prostate cancer genetic dataset. The empirical results demonstrate that Air-HOLP has improved performance over a large range of simulation settings. We provide R codes implementing the Air-HOLP feature screening method and integrating it into existing feature screening methods that utilize the HOLP formula.