$p$SVM: Soft-margin SVMs with $p$-norm Hinge Loss

TL;DR

This paper introduces pSVM, a novel soft-margin SVM with p-norm hinge loss, providing theoretical analysis, a new training algorithm, and empirical evidence of improved performance over traditional SVMs.

Contribution

The paper develops the theoretical properties, generalization bounds, and a specialized training algorithm for pSVMs, extending SVM capabilities with p-norm hinge loss.

Findings

pSVM achieves better classification accuracy on various datasets.

The pSMO algorithm effectively trains pSVMs with competitive efficiency.

Theoretical bounds support the generalization ability of pSVMs.

Abstract

Support Vector Machines (SVMs) based on hinge loss have been extensively discussed and applied to various binary classification tasks. These SVMs achieve a balance between margin maximization and the minimization of slack due to outliers. Although many efforts have been dedicated to enhancing the performance of SVMs with hinge loss, studies on $p$SVMs, soft-margin SVMs with $p$-norm hinge loss, remain relatively scarce. In this paper, we explore the properties, performance, and training algorithms of $p$SVMs. We first derive the generalization bound of $p$SVMs, then formulate the dual optimization problem, comparing it with the traditional approach. Furthermore, we discuss a generalized version of the Sequential Minimal Optimization (SMO) algorithm, $p$SMO, to train our $p$SVM model. Comparative experiments on various datasets, including binary and multi-class classification tasks, demonstrate the effectiveness and advantages of our $p$SVM model and the $p$SMO method. Code is available at https://github.com/CoderBak/pSVM.