Counterfactual Stress Testing for Image Classification Models

TL;DR

This paper introduces a causal generative model-based framework for realistic counterfactual stress testing of medical image classifiers, improving robustness evaluation under distribution shifts.

Contribution

It presents a novel counterfactual stress testing method using causal models to generate realistic images for assessing model robustness.

Findings

Counterfactual stress tests better predict out-of-distribution performance.

The method captures the direction and magnitude of performance changes.

It improves model ranking accuracy under distribution shifts.

Abstract

Deep learning models in medical imaging often fail when deployed in new clinical environments due to distribution shifts in demographics, scanner hardware, or acquisition protocols. A central challenge is underspecification, where models with similar validation performance exhibit divergent real-world failure modes. Although stress testing has emerged as a tool to assess this, current methods typically rely on simple, uninformed perturbations (e.g., brightness or contrast changes), which fail to capture clinically realistic variation and can overestimate robustness. In this work, we introduce a counterfactual stress testing framework based on causal generative models that create realistic "what if" images by intervening on attributes such as scanner type and patient sex while preserving anatomical identity, enabling controlled and semantically meaningful evaluation under targeted distribution shifts. Across two imaging modalities (chest X-ray and mammography), three model architectures, and multiple shift scenarios, we show that counterfactual stress tests provide a substantially more accurate proxy for real out-of-distribution performance than classical perturbations, capturing the direction and relative magnitude of performance changes as well as model ranking. These results suggest that causal generative models can serve as practical simulators for robustness assessment, offering a more reliable basis for evaluating medical AI systems prior to deployment.