$L_1$-norm Regularized Indefinite Kernel Logistic Regression

TL;DR

This paper introduces an $L_1$-norm regularized indefinite kernel logistic regression model that improves interpretability and accuracy by promoting sparsity, with a novel optimization algorithm handling the nonconvex, nonsmooth problem.

Contribution

It extends existing indefinite kernel logistic regression by incorporating $L_1$ regularization and develops an efficient proximal linearized algorithm for the complex optimization.

Findings

Outperforms existing methods in accuracy on benchmark datasets

Achieves higher sparsity leading to better interpretability

Demonstrates robustness across multiple domains

Abstract

Kernel logistic regression (KLR) is a powerful classification method widely applied across diverse domains. In many real-world scenarios, indefinite kernels capture more domain-specific structural information than positive definite kernels. This paper proposes a novel $L_1$-norm regularized indefinite kernel logistic regression (RIKLR) model, which extends the existing IKLR framework by introducing sparsity via an $L_1$-norm penalty. The introduction of this regularization enhances interpretability and generalization while introducing nonsmoothness and nonconvexity into the optimization landscape. To address these challenges, a theoretically grounded and computationally efficient proximal linearized algorithm is developed. Experimental results on multiple benchmark datasets demonstrate the superior performance of the proposed method in terms of both accuracy and sparsity.