PERFEX: Classifier Performance Explanations for Trustworthy AI Systems

TL;DR

PERFEX is a novel meta tree-based method that explains the conditions under which a classifier performs well or poorly, enhancing trustworthiness and decision support in real-world AI systems.

Contribution

The paper introduces PERFEX, a meta tree algorithm that predicts and explains classifier performance conditions, addressing a gap in existing explanation methods.

Findings

PERFEX achieves high meta prediction accuracy across classifiers and datasets.

It provides compact and informative performance explanations.

PERFEX is effective even when classifiers struggle to differentiate classes.

Abstract

Explainability of a classification model is crucial when deployed in real-world decision support systems. Explanations make predictions actionable to the user and should inform about the capabilities and limitations of the system. Existing explanation methods, however, typically only provide explanations for individual predictions. Information about conditions under which the classifier is able to support the decision maker is not available, while for instance information about when the system is not able to differentiate classes can be very helpful. In the development phase it can support the search for new features or combining models, and in the operational phase it supports decision makers in deciding e.g. not to use the system. This paper presents a method to explain the qualities of a trained base classifier, called PERFormance EXplainer (PERFEX). Our method consists of a meta tree learning algorithm that is able to predict and explain under which conditions the base classifier has a high or low error or any other classification performance metric. We evaluate PERFEX using several classifiers and datasets, including a case study with urban mobility data. It turns out that PERFEX typically has high meta prediction performance even if the base classifier is hardly able to differentiate classes, while giving compact performance explanations.