An Interpretable Local Editing Model for Counterfactual Medical Image Generation

TL;DR

The paper introduces InstructX2X, an interpretable local editing model for counterfactual medical image generation that prevents unintended changes and provides visual explanations, advancing the utility of AI in medical imaging.

Contribution

It proposes a novel Region-Specific Editing approach with Guidance Maps for interpretability and introduces a new dataset, MIMIC-EDIT-INSTRUCTION, for counterfactual medical image generation.

Findings

Achieves state-of-the-art performance on evaluation metrics.

Successfully generates high-quality counterfactual chest X-ray images.

Provides inherently interpretable visual explanations.

Abstract

Counterfactual medical image generation have emerged as a critical tool for enhancing AI-driven systems in medical domain by answering "what-if" questions. However, existing approaches face two fundamental limitations: First, they fail to prevent unintended modifications, resulting collateral changes in demographic attributes when only disease features should be affected. Second, they lack interpretability in their editing process, which significantly limits their utility in real-world medical applications. To address these limitations, we present InstructX2X, a novel interpretable local editing model for counterfactual medical image generation featuring Region-Specific Editing. This approach restricts modifications to specific regions, effectively preventing unintended changes while simultaneously providing a Guidance Map that offers inherently interpretable visual explanations of the editing process. Additionally, we introduce MIMIC-EDIT-INSTRUCTION, a dataset for counterfactual medical image generation derived from expert-verified medical VQA pairs. Through extensive experiments, InstructX2X achieve state-of-the-art performance across all major evaluation metrics. Our model successfully generates high-quality counterfactual chest X-ray images along with interpretable explanations.