RaSE: A Variable Screening Framework via Random Subspace Ensembles

TL;DR

RaSE introduces a flexible variable screening framework using random subspace ensembles, capable of detecting complex predictor interactions and marginal effects, with proven theoretical properties and demonstrated effectiveness.

Contribution

The paper proposes the RaSE framework, a novel ensemble-based variable screening method that captures joint effects and interactions, extending beyond marginal screening techniques.

Findings

Enjoys sure screening property and rank consistency.

Effective in identifying variables with no marginal effect.

Performs well in simulations and real data applications.

Abstract

Variable screening methods have been shown to be effective in dimension reduction under the ultra-high dimensional setting. Most existing screening methods are designed to rank the predictors according to their individual contributions to the response. As a result, variables that are marginally independent but jointly dependent with the response could be missed. In this work, we propose a new framework for variable screening, Random Subspace Ensemble (RaSE), which works by evaluating the quality of random subspaces that may cover multiple predictors. This new screening framework can be naturally combined with any subspace evaluation criterion, which leads to an array of screening methods. The framework is capable to identify signals with no marginal effect or with high-order interaction effects. It is shown to enjoy the sure screening property and rank consistency. We also develop an iterative version of RaSE screening with theoretical support. Extensive simulation studies and real-data analysis show the effectiveness of the new screening framework.