The soft-margin Support Vector Machine with ordered weighted average

TL;DR

This paper introduces an extended SVM model that incorporates ordered weighted sums of deviations, enabling more flexible classification with kernel functions, and demonstrates its effectiveness through computational experiments.

Contribution

It presents a novel SVM extension using ordered weighted deviations, with new quadratic and mixed-integer formulations, enhancing classification flexibility.

Findings

The new model generalizes classical SVMs by including ordered weighted deviations.

Kernel-based versions of the model perform well in experiments.

Computational results show improved predictive performance over existing SVMs.

Abstract

This paper deals with an extension of the Support Vector Machine (SVM) for classification problems where, in addition to maximize the margin, i.e., the width of strip defined by the two supporting hyperplanes, the minimum of the ordered weighted sum of the deviations of miclassified individuals is considered. Since the ordered weighted sum includes as particular case the sum of these deviations, the classical SVM model is a particular case of the model under study. A quadratic continuous formulation for the case in which weights are sorted in non-decreasing order is introduced, and a mixed integer quadratic formulation for general weights is presented. In both cases, we show that these formulations allow us the use of kernel functions to construct non linear classifiers. Besides, we report some computational results about the predictive performance of the introduced approach (also in its kernel version) in comparison with other SVM models existing in the literature.