Learning Support and Trivial Prototypes for Interpretable Image Classification

TL;DR

This paper introduces ST-ProtoPNet, a model that learns support prototypes near decision boundaries to enhance interpretability and accuracy in image classification, outperforming previous protoype-based methods.

Contribution

It proposes a novel support prototype learning approach for ProtoPNet, aligning with SVM theory, and develops ST-ProtoPNet to improve classification and interpretability.

Findings

ST-ProtoPNet achieves state-of-the-art accuracy on benchmark datasets.

Support prototypes are better localized in objects of interest.

The method enhances both interpretability and classification performance.

Abstract

Prototypical part network (ProtoPNet) methods have been designed to achieve interpretable classification by associating predictions with a set of training prototypes, which we refer to as trivial prototypes because they are trained to lie far from the classification boundary in the feature space. Note that it is possible to make an analogy between ProtoPNet and support vector machine (SVM) given that the classification from both methods relies on computing similarity with a set of training points (i.e., trivial prototypes in ProtoPNet, and support vectors in SVM). However, while trivial prototypes are located far from the classification boundary, support vectors are located close to this boundary, and we argue that this discrepancy with the well-established SVM theory can result in ProtoPNet models with inferior classification accuracy. In this paper, we aim to improve the classification of ProtoPNet with a new method to learn support prototypes that lie near the classification boundary in the feature space, as suggested by the SVM theory. In addition, we target the improvement of classification results with a new model, named ST-ProtoPNet, which exploits our support prototypes and the trivial prototypes to provide more effective classification. Experimental results on CUB-200-2011, Stanford Cars, and Stanford Dogs datasets demonstrate that ST-ProtoPNet achieves state-of-the-art classification accuracy and interpretability results. We also show that the proposed support prototypes tend to be better localised in the object of interest rather than in the background region.