An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

TL;DR

This paper introduces an efficient ADMM-based algorithm for nonconvex penalized SVMs, enabling effective variable selection and classification with guaranteed convergence and low computational cost.

Contribution

It proposes a novel ADMM algorithm applicable to various nonconvex penalties, with proven convergence and superior performance over existing methods.

Findings

Algorithm achieves lower computational cost

Demonstrates superior accuracy on benchmark datasets

Guarantees convergence for the proposed method

Abstract

Support vector machines (SVMs) with sparsity-inducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconvexity. In this paper, we propose an efficient ADMM-based algorithm to the nonconvex penalized SVMs. The proposed algorithm covers a large class of commonly used nonconvex regularization terms including the smooth clipped absolute deviation (SCAD) penalty, minimax concave penalty (MCP), log-sum penalty (LSP) and capped-$\ell_1$ penalty. The computational complexity analysis shows that the proposed algorithm enjoys low computational cost. Moreover, the convergence of the proposed algorithm is guaranteed. Extensive experimental evaluations on five benchmark datasets demonstrate the superior performance of the proposed algorithm to other three state-of-the-art approaches.