Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models

TL;DR

This paper evaluates prototype-based AI models for image classification, highlighting their current limitations in human interpretability and proposing properties for more meaningful prototypes to improve trustworthiness and understanding.

Contribution

The paper introduces properties for human-interpretable prototypes, analyzes existing methods against these properties, and discusses future directions for truly interpretable prototype models.

Findings

Existing prototypes often lack meaningfulness for human analysis

User study shows many prototypes are not useful for understanding

Identifies missing links and potential applications for better prototypes

Abstract

Explaining black-box Artificial Intelligence (AI) models is a cornerstone for trustworthy AI and a prerequisite for its use in safety critical applications such that AI models can reliably assist humans in critical decisions. However, instead of trying to explain our models post-hoc, we need models which are interpretable-by-design built on a reasoning process similar to humans that exploits meaningful high-level concepts such as shapes, texture or object parts. Learning such concepts is often hindered by its need for explicit specification and annotation up front. Instead, prototype-based learning approaches such as ProtoPNet claim to discover visually meaningful prototypes in an unsupervised way. In this work, we propose a set of properties that those prototypes have to fulfill to enable human analysis, e.g. as part of a reliable model assessment case, and analyse such existing methods in the light of these properties. Given a 'Guess who?' game, we find that these prototypes still have a long way ahead towards definite explanations. We quantitatively validate our findings by conducting a user study indicating that many of the learnt prototypes are not considered useful towards human understanding. We discuss about the missing links in the existing methods and present a potential real-world application motivating the need to progress towards truly human-interpretable prototypes.