DiffExplainer: Unveiling Black Box Models Via Counterfactual Generation

TL;DR

DiffExplainer introduces a novel method for interpreting black box medical image models by generating counterfactual images, revealing influential features that affect predictions and improving understanding of model decisions.

Contribution

The paper presents a new agent-based approach for generating counterfactual images to explain black box models in medical imaging, addressing limitations of existing explanation methods.

Findings

Effective identification of influential features in medical images.

Improved interpretability of deep learning models in prognosis tasks.

Validated approach shows superiority over existing methods.

Abstract

In the field of medical imaging, particularly in tasks related to early disease detection and prognosis, understanding the reasoning behind AI model predictions is imperative for assessing their reliability. Conventional explanation methods encounter challenges in identifying decisive features in medical image classifications, especially when discriminative features are subtle or not immediately evident. To address this limitation, we propose an agent model capable of generating counterfactual images that prompt different decisions when plugged into a black box model. By employing this agent model, we can uncover influential image patterns that impact the black model's final predictions. Through our methodology, we efficiently identify features that influence decisions of the deep black box. We validated our approach in the rigorous domain of medical prognosis tasks, showcasing its efficacy and potential to enhance the reliability of deep learning models in medical image classification compared to existing interpretation methods. The code will be publicly available at https://github.com/ayanglab/DiffExplainer.