Stacked Confusion Reject Plots (SCORE)

TL;DR

SCORE introduces an intuitive visualization tool for understanding classifier uncertainty and rejection trade-offs, improving interpretability over traditional reject curves especially for non-experts.

Contribution

The paper proposes Stacked Confusion Reject Plots (SCORE), a new visualization method that enhances interpretability of classifier rejection behavior in critical applications.

Findings

SCORE provides clearer insights into classifier uncertainty.

The method is demonstrated on artificial Gaussian data.

Code implementation is available as a Python package.

Abstract

Machine learning is more and more applied in critical application areas like health and driver assistance. To minimize the risk of wrong decisions, in such applications it is necessary to consider the certainty of a classification to reject uncertain samples. An established tool for this are reject curves that visualize the trade-off between the number of rejected samples and classification performance metrics. We argue that common reject curves are too abstract and hard to interpret by non-experts. We propose Stacked Confusion Reject Plots (SCORE) that offer a more intuitive understanding of the used data and the classifier's behavior. We present example plots on artificial Gaussian data to document the different options of SCORE and provide the code as a Python package.