Tutorial on Implied Posterior Probability for SVMs

TL;DR

This tutorial explains how to compute and calibrate implied posterior probabilities for SVMs in binary classification, providing a method to estimate true class probabilities from SVM outputs.

Contribution

It introduces a detailed approach for estimating and calibrating implied posterior probabilities in SVMs, enhancing their interpretability for probabilistic tasks.

Findings

Method for computing implied posterior probabilities for SVMs.

Calibration of these probabilities using isotonic regression.

Improved probability estimates for binary classification with SVMs.

Abstract

Implied posterior probability of a given model (say, Support Vector Machines (SVM)) at a point $\bf{x}$ is an estimate of the class posterior probability pertaining to the class of functions of the model applied to a given dataset. It can be regarded as a score (or estimate) for the true posterior probability, which can then be calibrated/mapped onto expected (non-implied by the model) posterior probability implied by the underlying functions, which have generated the data. In this tutorial we discuss how to compute implied posterior probabilities of SVMs for the binary classification case as well as how to calibrate them via a standard method of isotonic regression.