Investigating the Failure Modes of the AUC metric and Exploring Alternatives for Evaluating Systems in Safety Critical Applications

TL;DR

This paper identifies limitations in the AUC metric for evaluating models in safety-critical applications and proposes three alternative metrics, revealing that larger models do not always excel in selective answering tasks.

Contribution

The paper uncovers specific limitations of AUC and introduces three new metrics to improve evaluation of models in safety-critical contexts.

Findings

AUC can be misleading for selective answering evaluation

New metrics provide different insights into model performance

Larger models do not always outperform smaller ones in safety-critical tasks

Abstract

With the increasing importance of safety requirements associated with the use of black box models, evaluation of selective answering capability of models has been critical. Area under the curve (AUC) is used as a metric for this purpose. We find limitations in AUC; e.g., a model having higher AUC is not always better in performing selective answering. We propose three alternate metrics that fix the identified limitations. On experimenting with ten models, our results using the new metrics show that newer and larger pre-trained models do not necessarily show better performance in selective answering. We hope our insights will help develop better models tailored for safety-critical applications.