Explaining Image Classification with Visual Debates
Avinash Kori, Ben Glocker, Francesca Toni

TL;DR
This paper introduces a novel debate-based framework called Visual Debates to explain image classifier decisions by modeling reasoning as a multiplayer zero-sum game, providing diverse and insightful explanations.
Contribution
It proposes a new debate framework that models classifier explanations as a multiplayer game, encouraging diverse reasoning and highlighting uncertainties in predictions.
Findings
Effective explanations on SHAPE, MNIST, and AFHQ datasets.
Demonstrates improved faithfulness and completeness of explanations.
Provides novel metrics for evaluating visual debates as explanations.
Abstract
An effective way to obtain different perspectives on any given topic is by conducting a debate, where participants argue for and against the topic. Here, we propose a novel debate framework for understanding and explaining a continuous image classifier's reasoning for making a particular prediction by modeling it as a multiplayer sequential zero-sum debate game. The contrastive nature of our framework encourages players to learn to put forward diverse arguments during the debates, picking up the reasoning trails missed by their opponents and highlighting any uncertainties in the classifier. Specifically, in our proposed setup, players propose arguments, drawn from the classifier's discretized latent knowledge, to support or oppose the classifier's decision. The resulting Visual Debates collect supporting and opposing features from the discretized latent space of the classifier, serving…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Explainable Artificial Intelligence (XAI) · Generative Adversarial Networks and Image Synthesis
