A Safe Screening Rule for Sparse Logistic Regression

TL;DR

This paper introduces Slores, a fast screening rule for sparse logistic regression that significantly reduces feature set size and computational cost, enhancing efficiency across various high-dimensional datasets.

Contribution

The paper presents Slores, a novel, solver-independent screening rule that efficiently identifies zero components in sparse logistic regression solutions, improving computational speed.

Findings

Slores outperforms existing screening rules in speed.

Slores reduces the number of features to optimize.

Efficiency improved by an order of magnitude.

Abstract

The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection. Although many recent efforts have been devoted to its efficient implementation, its application to high dimensional data still poses significant challenges. In this paper, we present a fast and effective sparse logistic regression screening rule (Slores) to identify the 0 components in the solution vector, which may lead to a substantial reduction in the number of features to be entered to the optimization. An appealing feature of Slores is that the data set needs to be scanned only once to run the screening and its computational cost is negligible compared to that of solving the sparse logistic regression problem. Moreover, Slores is independent of solvers for sparse logistic regression, thus Slores can be integrated with any existing solver to improve the efficiency. We have evaluated Slores using high-dimensional data sets from different applications. Extensive experimental results demonstrate that Slores outperforms the existing state-of-the-art screening rules and the efficiency of solving sparse logistic regression is improved by one magnitude in general.