Support Vector Machine Classifier with Rescaled Huberized Pinball Loss

TL;DR

This paper introduces RHPSVM, a novel SVM model using a rescaled Huberized pinball loss that improves robustness to outliers and resampling stability, with strong theoretical guarantees and superior performance on various datasets.

Contribution

The paper proposes a new loss function and SVM model, RHPSVM, with theoretical analysis and an efficient optimization algorithm, enhancing robustness and flexibility over traditional SVMs.

Findings

RHPSVM outperforms existing SVMs on noisy and noise-free data.

RHPSVM is effective in high-dimensional small-sample scenarios.

The model has strong theoretical properties including Bayesian conformity and generalization bounds.

Abstract

Support vector machines are widely used in machine learning classification tasks, but traditional SVM models suffer from sensitivity to outliers and instability in resampling, which limits their performance in practical applications. To address these issues, this paper proposes a novel rescaled Huberized pinball loss function with asymmetric, non-convex, and smooth properties. Based on this loss function, we develop a corresponding SVM model called RHPSVM (Rescaled Huberized Pinball Loss Support Vector Machine). Theoretical analyses demonstrate that RHPSVM conforms to Bayesian rules, has a strict generalization error bound, a bounded influence function, and controllable optimality conditions, ensuring excellent classification accuracy, outlier insensitivity, and resampling stability. Additionally, RHPSVM can be extended to various advanced SVM variants by adjusting parameters, enhancing its flexibility. We transform the non-convex optimization problem of RHPSVM into a series of convex subproblems using the concave-convex procedure (CCCP) and solve it with the ClipDCD algorithm, which is proven to be convergent. Experimental results on simulated data, UCI datasets, and small-sample crop leaf image classification tasks show that RHPSVM outperforms existing SVM models in both noisy and noise-free scenarios, especially in handling high-dimensional small-sample data.