Using SVM to Estimate and Predict Binary Choice Models

TL;DR

This paper demonstrates that SVM can be used to consistently estimate the slope parameters of binary choice models, showing asymptotic equivalence to logistic regression under certain conditions, especially with imbalanced classes.

Contribution

It establishes the theoretical connection between SVM and binary choice models, providing conditions under which SVM slope estimates are consistent and comparable to logistic regression.

Findings

SVM slope estimator is consistent under certain conditions.

SVM asymptotically parallels QMLE for binary outcomes.

Finite-sample performance varies with distributions.

Abstract

The support vector machine (SVM) has an asymptotic behavior that parallels that of the quasi-maximum likelihood estimator (QMLE) for binary outcomes generated by a binary choice model (BCM), although it is not a QMLE. We show that, under the linear conditional mean condition for covariates given the systematic component used in the QMLE slope consistency literature, the slope of the separating hyperplane given by the SVM consistently estimates the BCM slope parameter, as long as the class weight is used as required when binary outcomes are severely imbalanced. The SVM slope estimator is asymptotically equivalent to that of logistic regression in this sense. The finite-sample performance of the two estimators can be quite distinct depending on the distributions of covariates and errors, but neither dominates the other. The intercept parameter of the BCM can be consistently estimated once a consistent estimator of its slope parameter is obtained.