Multiclass threshold-based classification and model evaluation

TL;DR

This paper proposes a novel threshold-based framework for multiclass classification that improves prediction accuracy by optimizing thresholds in a geometric simplex space and introduces a new ROC analysis method called ROC clouds.

Contribution

It introduces a geometric thresholding approach for multiclass classification and develops a new ROC analysis method called ROC clouds for better model evaluation.

Findings

Multidimensional threshold tuning improves network performance.

The proposed ROC clouds provide a coherent alternative to OvR curves.

Threshold optimization enhances classification accuracy across datasets.

Abstract

In this paper, we introduce a threshold-based framework for multiclass classification that generalizes the standard argmax rule. This is done by replacing the probabilistic interpretation of softmax outputs with a geometric one on the multidimensional simplex, where the classification depends on a multidimensional threshold. This change of perspective enables for any trained classification network an \textit{a posteriori} optimization of the classification score by means of threshold tuning, as usually carried out in the binary setting, thus allowing for a further refinement of the prediction capability of any network. Our experiments show indeed that multidimensional threshold tuning yields performance improvements across various networks and datasets. Moreover, we derive a multiclass ROC analysis based on \emph{ROC clouds} -- the attainable (FPR,TPR) operating points induced by a single multiclass threshold -- and summarize them via a \emph{Distance From Point} (DFP) score to $(0,1)$. This yields a coherent alternative to standard One-vs-Rest (OvR) curves and aligns with the observed tuning gains.