LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision

TL;DR

LucidPPN enhances interpretability in prototypical parts networks by separating color from other features, clarifying decision factors, and aligning prototypes with semantic object parts, thus improving user understanding.

Contribution

The paper introduces LucidPPN, a novel network that separates color and non-color features for clearer, more intuitive explanations in visual classification tasks.

Findings

Achieves comparable accuracy to baseline methods.

Produces less ambiguous and more interpretable prototypes.

Effectively distinguishes decision features like color, shape, and texture.

Abstract

Prototypical parts networks combine the power of deep learning with the explainability of case-based reasoning to make accurate, interpretable decisions. They follow the this looks like that reasoning, representing each prototypical part with patches from training images. However, a single image patch comprises multiple visual features, such as color, shape, and texture, making it difficult for users to identify which feature is important to the model. To reduce this ambiguity, we introduce the Lucid Prototypical Parts Network (LucidPPN), a novel prototypical parts network that separates color prototypes from other visual features. Our method employs two reasoning branches: one for non-color visual features, processing grayscale images, and another focusing solely on color information. This separation allows us to clarify whether the model's decisions are based on color, shape, or texture. Additionally, LucidPPN identifies prototypical parts corresponding to semantic parts of classified objects, making comparisons between data classes more intuitive, e.g., when two bird species might differ primarily in belly color. Our experiments demonstrate that the two branches are complementary and together achieve results comparable to baseline methods. More importantly, LucidPPN generates less ambiguous prototypical parts, enhancing user understanding.