On Regularized Sparse Logistic Regression

TL;DR

This paper introduces a unified framework for solving sparse logistic regression with both convex and nonconvex regularization, ensuring convergence and demonstrating effective feature selection and classification with lower computational costs.

Contribution

It proposes a versatile algorithmic framework that extends to nonconvex regularization, with a novel line search ensuring convergence.

Findings

Effective feature selection and classification on real datasets.

Lower computational cost compared to existing methods.

Successful extension to nonconvex regularization.

Abstract

Sparse logistic regression is for classification and feature selection simultaneously. Although many studies have been done to solve $\ell_1$-regularized logistic regression, there is no equivalently abundant work on solving sparse logistic regression with nonconvex regularization term. In this paper, we propose a unified framework to solve $\ell_1$-regularized logistic regression, which can be naturally extended to nonconvex regularization term, as long as certain requirement is satisfied. In addition, we also utilize a different line search criteria to guarantee monotone convergence for various regularization terms. Empirical experiments on binary classification tasks with real-world datasets demonstrate our proposed algorithms are capable of performing classification and feature selection effectively at a lower computational cost.