Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays

TL;DR

This paper explores using concept vectors in generative models to produce interpretable counterfactual explanations for chest X-ray images, aiming to improve clinical interpretability of AI models.

Contribution

It introduces a method to map clinical concepts into the latent space of autoencoders for generating concept-based counterfactuals without explicit labels.

Findings

Concept vectors are stable across datasets.

Counterfactuals can exaggerate or reduce clinical features.

Promising results on large pathologies like cardiomegaly.

Abstract

An essential step in deploying medical imaging models is ensuring alignment with clinical knowledge and interpretability. We focus on mapping clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs). Using a simple reconstruction autoencoder, we link user-defined concepts to image-level features without explicit label training. The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features. By traversing latent space along concept directions, we produce counterfactuals that exaggerate or reduce specific clinical features. Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging due to reconstruction limits. Although not outperforming baselines, this approach offers a path toward interpretable, concept-based explanations aligned with clinical knowledge.