Greedy Forward Regression for Variable Screening

TL;DR

This paper introduces a new variable screening method that generalizes forward regression and sure independence screening, reducing computational burden while maintaining high accuracy in identifying relevant variables in high-dimensional data.

Contribution

It proposes a simple, unified screening method that combines the strengths of FR and SIS, preserving sure screening property and improving efficiency.

Findings

Method retains sure screening property.

It discovers relevant variables in fewer steps.

Simulation and real data show excellent performance.

Abstract

Two popular variable screening methods under the ultra-high dimensional setting with the desirable sure screening property are the sure independence screening (SIS) and the forward regression (FR). Both are classical variable screening methods and recently have attracted greater attention under the new light of high-dimensional data analysis. We consider a new and simple screening method that incorporates multiple predictors in each step of forward regression, with decision on which variables to incorporate based on the same criterion. If only one step is carried out, it actually reduces to the SIS. Thus it can be regarded as a generalization and unification of the FR and the SIS. More importantly, it preserves the sure screening property and has similar computational complexity as FR in each step, yet it can discover the relevant covariates in fewer steps. Thus, it reduces the computational burden of FR drastically while retaining advantages of the latter over SIS. Furthermore, we show that it can find all the true variables if the number of steps taken is the same as the correct model size, even when using the original FR. An extensive simulation study and application to two real data examples demonstrate excellent performance of the proposed method.