Facility Locations Utility for Uncovering Classifier Overconfidence

TL;DR

This paper introduces a facility locations utility model and greedy algorithm to identify overconfident misclassifications in classifiers, improving the detection of model deficiencies without labeled data.

Contribution

The paper presents a novel utility model and query algorithm specifically designed to uncover overconfident unknown unknowns in classifiers, addressing a gap in existing methods.

Findings

The proposed method outperforms previous approaches in discovering overconfident errors.

The utility model effectively guides queries to identify model overconfidence.

Empirical results demonstrate superior performance in real-world scenarios.

Abstract

Assessing the predictive accuracy of black box classifiers is challenging in the absence of labeled test datasets. In these scenarios we may need to rely on a human oracle to evaluate individual predictions; presenting the challenge to create query algorithms to guide the search for points that provide the most information about the classifier's predictive characteristics. Previous works have focused on developing utility models and query algorithms for discovering unknown unknowns --- misclassifications with a predictive confidence above some arbitrary threshold. However, if misclassifications occur at the rate reflected by the confidence values, then these search methods reveal nothing more than a proper assessment of predictive certainty. We are unable to properly mitigate the risks associated with model deficiency when the model's confidence in prediction exceeds the actual model accuracy. We propose a facility locations utility model and corresponding greedy query algorithm that instead searches for overconfident unknown unknowns. Through robust empirical experiments we demonstrate that the greedy query algorithm with the facility locations utility model consistently results in oracle queries with superior performance in discovering overconfident unknown unknowns than previous methods.