What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?

TL;DR

This paper introduces SoftmaxCorr, a new measure based on Softmax outputs that effectively ranks classifier performance across diverse test conditions, including out-of-distribution data.

Contribution

We propose SoftmaxCorr, a novel correlation-based measure that predicts classifier performance using Softmax outputs and class correlation matrices.

Findings

SoftmaxCorr accurately forecasts model performance on ID and OOD datasets.

Conventional Softmax probability has utility in certain OOD contexts.

SoftmaxCorr outperforms traditional uncertainty metrics in ranking classifiers.

Abstract

This work aims to develop a measure that can accurately rank the performance of various classifiers when they are tested on unlabeled data from out-of-distribution (OOD) distributions. We commence by demonstrating that conventional uncertainty metrics, notably the maximum Softmax prediction probability, possess inherent utility in forecasting model generalization across certain OOD contexts. Building on this insight, we introduce a new measure called Softmax Correlation (SoftmaxCorr). It calculates the cosine similarity between a class-class correlation matrix, constructed from Softmax output vectors across an unlabeled test dataset, and a predefined reference matrix that embodies ideal class correlations. A high resemblance of predictions to the reference matrix signals that the model delivers confident and uniform predictions across all categories, reflecting minimal uncertainty and confusion. Through rigorous evaluation across a suite of datasets, including ImageNet, CIFAR-10, and WILDS, we affirm the predictive validity of SoftmaxCorr in accurately forecasting model performance within both in-distribution (ID) and OOD settings. Furthermore, we discuss the limitations of our proposed measure and suggest avenues for future research.