Mask-Guided Attention Regulation for Anatomically Consistent Counterfactual CXR Synthesis

TL;DR

This paper introduces an attention regulation framework for diffusion-based chest X-ray synthesis that improves anatomical consistency and controllability of pathological modifications, addressing structural drift and weak lesion signals.

Contribution

It proposes novel attention regularization and pathology-guided modules that confine structural interactions and enhance lesion localization during diffusion-based CXR editing.

Findings

Improved anatomical consistency in synthesized CXRs.

More precise and controllable lesion editing.

Enhanced performance in downstream diagnostic tasks.

Abstract

Counterfactual generation for chest X-rays (CXR) aims to simulate plausible pathological changes while preserving patient-specific anatomy. However, diffusion-based editing methods often suffer from structural drift, where stable anatomical semantics propagate globally through attention and distort non-target regions, and unstable pathology expression, since subtle and localized lesions induce weak and noisy conditioning signals. We present an inference-time attention regulation framework for reliable counterfactual CXR synthesis. An anatomy-aware attention regularization module gates self-attention and anatomy-token cross-attention with organ masks, confining structural interactions to anatomical ROIs and reducing unintended distortions. A pathology-guided module enhances pathology-token cross-attention within target lung regions during early denoising and performs lightweight latent corrections driven by an attention-concentration energy, enabling controllable lesion localization and extent. Extensive evaluations on CXR datasets show improved anatomical consistency and more precise, controllable pathological edits compared with standard diffusion editing, supporting localized counterfactual analysis and data augmentation for downstream tasks.