Robust Classification via Support Vector Machines

TL;DR

This paper introduces two probabilistic methods to enhance Support Vector Machine classifiers, making them more robust against feature data uncertainty, with demonstrated efficiency and effectiveness on synthetic and real data.

Contribution

It proposes two novel robust SVM classifiers based on probabilistic arguments, addressing data uncertainty more effectively than traditional methods.

Findings

Both classifiers are computationally efficient.

They outperform standard SVMs on uncertain data.

The methods have limitations depending on data conditions.

Abstract

Classification models are very sensitive to data uncertainty, and finding robust classifiers that are less sensitive to data uncertainty has raised great interest in the machine learning literature. This paper aims to construct robust \emph{Support Vector Machine} classifiers under feature data uncertainty via two probabilistic arguments. The first classifier, \emph{Single Perturbation}, reduces the local effect of data uncertainty with respect to one given feature and acts as a local test that could confirm or refute the presence of significant data uncertainty for that particular feature. The second classifier, \emph{Extreme Empirical Loss}, aims to reduce the aggregate effect of data uncertainty with respect to all features, which is possible via a trade-off between the number of prediction model violations and the size of these violations. Both methodologies are computationally efficient and our extensive numerical investigation highlights the advantages and possible limitations of the two robust classifiers on synthetic and real-life data.