Counterfactual Visual Explanations
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee

TL;DR
This paper introduces a method for generating counterfactual visual explanations by identifying and replacing regions in images to show how the system's predicted class can change, enhancing interpretability and human understanding.
Contribution
The work presents a novel technique for producing counterfactual visual explanations that improve interpretability and aid human learning in image classification tasks.
Findings
Counterfactual explanations improve human ability to distinguish bird species.
The method provides qualitative insights into model decision boundaries.
Users trained with explanations perform better in classification tasks.
Abstract
In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image for which a vision system predicts class , a counterfactual visual explanation identifies how could change such that the system would output a different specified class . To do this, we select a 'distractor' image that the system predicts as class and identify spatial regions in and such that replacing the identified region in with the identified region in would push the system towards classifying as . We apply our approach to multiple image classification datasets generating qualitative results showcasing the interpretability and discriminativeness of our counterfactual explanations. To explore the effectiveness of our explanations in teaching humans, we present machine teaching experiments for the task of fine-grained bird…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsExplainable Artificial Intelligence (XAI) · Cell Image Analysis Techniques · Machine Learning and Data Classification
MethodsInterpretability
