Comprehensive Evaluation of Prototype Neural Networks

TL;DR

This paper provides a comprehensive evaluation of prominent prototype models for explainable AI, introducing new interpretability metrics and testing on diverse datasets, supported by an open-source toolkit.

Contribution

It introduces new metrics for interpretability, evaluates multiple prototype models across diverse datasets, and offers an open-source library for analysis and extension.

Findings

Prototype models vary significantly in interpretability and performance.

New metrics provide deeper insights into model explanations.

Open-source toolkit facilitates broader adoption and customization.

Abstract

Prototype models are an important method for explainable artificial intelligence (XAI) and interpretable machine learning. In this paper, we perform an in-depth analysis of a set of prominent prototype models including ProtoPNet, ProtoPool and PIPNet. For their assessment, we apply a comprehensive set of metrics. In addition to applying standard metrics from literature, we propose several new metrics to further complement the analysis of model interpretability. In our experimentation, we apply the set of prototype models on a diverse set of datasets including fine-grained classification, Non-IID settings and multi-label classification to further contrast the performance. Furthermore, we also provide our code as an open-source library (https://github.com/uos-sis/quanproto), which facilitates simple application of the metrics itself, as well as extensibility -- providing the option for easily adding new metrics and models.