Positional Segmentor-Guided Counterfactual Fine-Tuning for Spatially Localized Image Synthesis

TL;DR

This paper introduces Positional Segmentor-Guided Counterfactual Fine-Tuning, a method for generating localized, anatomically coherent image modifications by subdividing structures into regions, improving spatial control in medical image synthesis.

Contribution

It extends previous segmentor-guided counterfactual methods by enabling region-specific, localized modifications through subdivision of structures into regions for more precise image synthesis.

Findings

Generates realistic, region-specific modifications in coronary CT images.

Provides finer spatial control for disease progression modeling.

Outperforms existing global intervention methods.

Abstract

Counterfactual image generation enables controlled data augmentation, bias mitigation, and disease modeling. However, existing methods guided by external classifiers or regressors are limited to subject-level factors (e.g., age) and fail to produce localized structural changes, often resulting in global artifacts. Pixel-level guidance using segmentation masks has been explored, but requires user-defined counterfactual masks, which are tedious and impractical. Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT) addressed this by using segmentation-derived measurements to supervise structure-specific variables, yet it remains restricted to global interventions. We propose Positional Seg-CFT, which subdivides each structure into regional segments and derives independent measurements per region, enabling spatially localized and anatomically coherent counterfactuals. Experiments on coronary CT angiography show that Pos-Seg-CFT generates realistic, region-specific modifications, providing finer spatial control for modeling disease progression.