Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations

TL;DR

This paper introduces a benchmark and metrics to evaluate spatial misalignment in prototypical parts-based networks, proposes a compensation method, and demonstrates its effectiveness through extensive empirical studies.

Contribution

It presents the first interpretability benchmark for spatial misalignment and a novel compensation method for improving explanations in prototypical parts networks.

Findings

Benchmark effectively quantifies spatial misalignment.

Proposed compensation improves interpretability.

Empirical results validate the benchmark and method.

Abstract

Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.