DeCoDEx: Confounder Detector Guidance for Improved Diffusion-based Counterfactual Explanations

TL;DR

DeCoDEx introduces a framework that uses an external artifact detector to guide diffusion-based counterfactual image generation, improving explainability and bias mitigation in medical image classifiers.

Contribution

The paper proposes a novel method combining artifact detection with diffusion models to enhance counterfactual explanations and reduce bias in deep learning classifiers.

Findings

Successfully generates counterfactual images that alter causal markers while ignoring artifacts.

Improves classifier performance on underrepresented, out-of-distribution groups.

Demonstrates effectiveness on CheXpert dataset with synthetic and real artifacts.

Abstract

Deep learning classifiers are prone to latching onto dominant confounders present in a dataset rather than on the causal markers associated with the target class, leading to poor generalization and biased predictions. Although explainability via counterfactual image generation has been successful at exposing the problem, bias mitigation strategies that permit accurate explainability in the presence of dominant and diverse artifacts remain unsolved. In this work, we propose the DeCoDEx framework and show how an external, pre-trained binary artifact detector can be leveraged during inference to guide a diffusion-based counterfactual image generator towards accurate explainability. Experiments on the CheXpert dataset, using both synthetic artifacts and real visual artifacts (support devices), show that the proposed method successfully synthesizes the counterfactual images that change the causal pathology markers associated with Pleural Effusion while preserving or ignoring the visual artifacts. Augmentation of ERM and Group-DRO classifiers with the DeCoDEx generated images substantially improves the results across underrepresented groups that are out of distribution for each class. The code is made publicly available at https://github.com/NimaFathi/DeCoDEx.