CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanations

TL;DR

CoLa-DCE introduces a novel method for generating transparent, concept-guided counterfactual explanations for image classifiers, enhancing interpretability by controlling feature changes and visualizing their spatial localization.

Contribution

It presents a new diffusion-based approach that improves the transparency and granularity of counterfactual explanations in computer vision models.

Findings

Enhanced control over concept selection and spatial conditioning.

Counterfactuals with minimal feature changes and better interpretability.

Insights into model errors like misclassification cases.

Abstract

Recent advancements in generative AI have introduced novel prospects and practical implementations. Especially diffusion models show their strength in generating diverse and, at the same time, realistic features, positioning them well for generating counterfactual explanations for computer vision models. Answering "what if" questions of what needs to change to make an image classifier change its prediction, counterfactual explanations align well with human understanding and consequently help in making model behavior more comprehensible. Current methods succeed in generating authentic counterfactuals, but lack transparency as feature changes are not directly perceivable. To address this limitation, we introduce Concept-guided Latent Diffusion Counterfactual Explanations (CoLa-DCE). CoLa-DCE generates concept-guided counterfactuals for any classifier with a high degree of control regarding concept selection and spatial conditioning. The counterfactuals comprise an increased granularity through minimal feature changes. The reference feature visualization ensures better comprehensibility, while the feature localization provides increased transparency of "where" changed "what". We demonstrate the advantages of our approach in minimality and comprehensibility across multiple image classification models and datasets and provide insights into how our CoLa-DCE explanations help comprehend model errors like misclassification cases.