A binary-response regression model based on support vector machines

TL;DR

This paper introduces a probabilistic binary-response regression model derived from the support vector machine's optimization framework, enabling probabilistic inference and demonstrating strong theoretical and empirical performance.

Contribution

It develops a new probabilistic regression model based on SVM optimization, with proven statistical properties and practical applications.

Findings

MLE exists under weak conditions

Model is consistent and asymptotically normal

Effective in spam detection and water access prediction

Abstract

The soft-margin support vector machine (SVM) is a ubiquitous tool for prediction of binary-response data. However, the SVM is characterized entirely via a numerical optimization problem, rather than a probability model, and thus does not directly generate probabilistic inferential statements as outputs. We consider a probabilistic regression model for binary-response data that is based on the optimization problem that characterizes the SVM. Under weak regularity assumptions, we prove that the maximum likelihood estimate (MLE) of our model exists, and that it is consistent and asymptotically normal. We further assess the performance of our model via simulation studies, and demonstrate its use in real data applications regarding spam detection and well water access.