Toward Simple and Robust Contrastive Explanations for Image Classification by Leveraging Instance Similarity and Concept Relevance

TL;DR

This paper proposes a concept-based contrastive explanation method for image classification that leverages instance similarity and concept relevance, demonstrating improved interpretability and robustness under image augmentations.

Contribution

It introduces a novel approach combining concept relevance and instance similarity to generate more interpretable contrastive explanations for image classifiers.

Findings

Higher concept relevance yields shorter, less complex explanations.

Explanations vary in robustness under image augmentations.

Concept relevance correlates with explanation simplicity.

Abstract

Understanding why a classification model prefers one class over another for an input instance is the challenge of contrastive explanation. This work implements concept-based contrastive explanations for image classification by leveraging the similarity of instance embeddings and relevance of human-understandable concepts used by a fine-tuned deep learning model. Our approach extracts concepts with their relevance score, computes contrasts for similar instances, and evaluates the resulting contrastive explanations based on explanation complexity. Robustness is tested for different image augmentations. Two research questions are addressed: (1) whether explanation complexity varies across different relevance ranges, and (2) whether explanation complexity remains consistent under image augmentations such as rotation and noise. The results confirm that for our experiments higher concept relevance leads to shorter, less complex explanations, while lower relevance results in longer, more diffuse explanations. Additionally, explanations show varying degrees of robustness. The discussion of these findings offers insights into the potential of building more interpretable and robust AI systems.