Calibrated simplex-mapping classification

TL;DR

This paper introduces a new multi-class classification method that creates a well-calibrated predictor by mapping data into a simplex-induced latent space and extending it with regression, ensuring reliable confidence levels.

Contribution

It presents a novel two-step classification approach using a simplex-based latent space and regression extension, with theoretical guarantees and extensive benchmarking.

Findings

Achieves well-calibrated probability estimates.

Performs competitively on synthetic and real datasets.

Provides theoretical analysis of calibration properties.

Abstract

We propose a novel methodology for general multi-class classification in arbitrary feature spaces, which results in a potentially well-calibrated classifier. Calibrated classifiers are important in many applications because, in addition to the prediction of mere class labels, they also yield a confidence level for each of their predictions. In essence, the training of our classifier proceeds in two steps. In a first step, the training data is represented in a latent space whose geometry is induced by a regular $(n-1)$-dimensional simplex, $n$ being the number of classes. We design this representation in such a way that it well reflects the feature space distances of the datapoints to their own- and foreign-class neighbors. In a second step, the latent space representation of the training data is extended to the whole feature space by fitting a regression model to the transformed data. With this latent-space representation, our calibrated classifier is readily defined. We rigorously establish its core theoretical properties and benchmark its prediction and calibration properties by means of various synthetic and real-world data sets from different application domains.