Linear Maximum Margin Classifier for Learning from Uncertain Data

TL;DR

This paper introduces SVM-GSU, a maximum margin classifier that incorporates data uncertainty modeled as Gaussian distributions, improving classification robustness on uncertain and noisy data.

Contribution

It reformulates the SVM framework to handle Gaussian-distributed data with uncertainty, resulting in a convex optimization problem solved via stochastic gradient descent.

Findings

Effective on synthetic and real datasets

Outperforms traditional SVM in uncertain data scenarios

Efficient primal optimization approach

Abstract

In this paper, we propose a maximum margin classifier that deals with uncertainty in data input. More specifically, we reformulate the SVM framework such that each training example can be modeled by a multi-dimensional Gaussian distribution described by its mean vector and its covariance matrix -- the latter modeling the uncertainty. We address the classification problem and define a cost function that is the expected value of the classical SVM cost when data samples are drawn from the multi-dimensional Gaussian distributions that form the set of the training examples. Our formulation approximates the classical SVM formulation when the training examples are isotropic Gaussians with variance tending to zero. We arrive at a convex optimization problem, which we solve efficiently in the primal form using a stochastic gradient descent approach. The resulting classifier, which we name SVM with Gaussian Sample Uncertainty (SVM-GSU), is tested on synthetic data and five publicly available and popular datasets; namely, the MNIST, WDBC, DEAP, TV News Channel Commercial Detection, and TRECVID MED datasets. Experimental results verify the effectiveness of the proposed method.