Seeing What Shouldn't Be There: Counterfactual GANs for Medical Image Attribution

TL;DR

This paper introduces a counterfactual GAN-based method for medical image attribution that generates plausible counterfactual explanations and visualizes influential objects for improved radiological insights.

Contribution

It proposes a novel counterfactual explanation technique using GANs with cyclical loss, addressing limitations of existing methods and enhancing interpretability in medical imaging.

Findings

Method effective on synthetic, tuberculosis, and BraTS datasets.

Generated counterfactuals are plausible and improve interpretability.

Baseline results demonstrate the approach's potential.

Abstract

Ascription of an image gives insights into the objects that influence the classification of the whole image or its pixels towards a specific category. These insights help radiologists to visualize deformities in medical imaging. Most of the existing visualization techniques are based on discriminative models and highlight regions of the input image participating in the decision-making of a classifier. However, these approaches do not take all noticeable objects into account as their objective is to classify the input by using a minimal set of discriminative features. To overcome the issue, a counterfactual explanation (CX) based class-oriented feature attribution method is proposed. A counterfactual explanation (CX) explicates a causal reasoning process of the form: "if X had not happened, then Y would not have happened". The method is built on generative adversarial networks (GANs) with a cyclical-consistent loss function. We evaluate our method on three datasets: synthetic, tuberculosis and BraTS. All experiments confirm the efficacy of the proposed method. This study also highlighted the limitations of existing counterfactual explanation techniques in producing plausible counterfactual instances (CIs). Accompanying CXs with believable CIs thus provides self-explanatory analogy-based explanations. To this end, a CI generation method is proposed. Also, a novel technique is used to evaluate the quality of CI. The baseline results are produced on the BraTS dataset.