Safe Screening for Logistic Regression with $\ell_0$-$\ell_2$ Regularization

TL;DR

This paper introduces safe screening rules for logistic regression with combined 0- 0 regularization, enabling the removal of irrelevant features before solving, which accelerates computation significantly.

Contribution

It proposes novel safe screening rules based on Fenchel dual bounds for 0- 0 regularized logistic regression, improving efficiency over existing methods.

Findings

High percentage of features safely removed pre-solution

Significant computational speed-up observed in experiments

Effective in both real and synthetic datasets

Abstract

In logistic regression, it is often desirable to utilize regularization to promote sparse solutions, particularly for problems with a large number of features compared to available labels. In this paper, we present screening rules that safely remove features from logistic regression with $\ell_0-\ell_2$ regularization before solving the problem. The proposed safe screening rules are based on lower bounds from the Fenchel dual of strong conic relaxations of the logistic regression problem. Numerical experiments with real and synthetic data suggest that a high percentage of the features can be effectively and safely removed apriori, leading to substantial speed-up in the computations.