Newton Method for Soft Quadratic Surface Support Vector Machine with 0-1 Loss Function

TL;DR

This paper introduces a novel Newton method for a non-convex, kernel-free support vector machine model with 0-1 loss, achieving high accuracy efficiently.

Contribution

It establishes the first second-order optimality conditions and develops a Newton method with quadratic convergence for the $L_{0/1}$-SQSSVM model.

Findings

The Newton method converges locally quadratically.

Numerical experiments show higher accuracy and less CPU time than existing methods.

The model effectively handles non-convex, discontinuous 0-1 loss in binary classification.

Abstract

A nonlinear kernel-free soft quadratic surface support vector machine model with 0-1 loss function ($L_{0/1}$-SQSSVM) is proposed for binary classification problems, which is non-convex discontinuous. We are devoted to establishing the first and the second-order optimality conditions for the $L_{0/1}$-SQSSVM. We establish a stationary equation using the properties of proximal operator of 0-1 loss function. We design a Newton method based on the stationary equation to solve $L_{0/1}$-SQSSVM model and prove that the Newton method has local quadratic convergence under the second-order sufficient condition. Numerical experience on artificial datasets and benchmark datasets demonstrate that the Newton method for $L_{0/1}$-SQSSVM achieves higher classification accuracy with less CPU time cost than other state-of-the-art methods.