Marginal empirical likelihood and sure independence feature screening

TL;DR

This paper introduces a novel feature screening method based on marginal empirical likelihood ratios, effective in high-dimensional settings, capable of incorporating estimator uncertainties, and adaptable to various models.

Contribution

It proposes a unified, less assumption-dependent feature screening procedure using empirical likelihood ratios, extending existing methods with uncertainty incorporation.

Findings

Effective in high-dimensional variable selection

Capable of incorporating estimator uncertainties

Applicable to broad model classes

Abstract

We study a marginal empirical likelihood approach in scenarios when the number of variables grows exponentially with the sample size. The marginal empirical likelihood ratios as functions of the parameters of interest are systematically examined, and we find that the marginal empirical likelihood ratio evaluated at zero can be used to differentiate whether an explanatory variable is contributing to a response variable or not. Based on this finding, we propose a unified feature screening procedure for linear models and the generalized linear models. Different from most existing feature screening approaches that rely on the magnitudes of some marginal estimators to identify true signals, the proposed screening approach is capable of further incorporating the level of uncertainties of such estimators. Such a merit inherits the self-studentization property of the empirical likelihood approach, and extends the insights of existing feature screening methods. Moreover, we show that our screening approach is less restrictive to distributional assumptions, and can be conveniently adapted to be applied in a broad range of scenarios such as models specified using general moment conditions. Our theoretical results and extensive numerical examples by simulations and data analysis demonstrate the merits of the marginal empirical likelihood approach.