TRACE: Temporally Reliable Anatomically-Conditioned 3D CT Generation with Enhanced Efficiency

TL;DR

TRACE is a novel framework for generating 3D medical images that ensures anatomical accuracy and temporal consistency while being computationally efficient, suitable for resource-limited clinical settings.

Contribution

It introduces a 2D diffusion-based approach with segmentation priors and optical flow for reliable 3D CT generation, improving over existing methods in fidelity and efficiency.

Findings

Balances computational cost and anatomical fidelity

Maintains spatiotemporal coherence in generated images

Effective for clinical data augmentation

Abstract

3D medical image generation is essential for data augmentation and patient privacy, calling for reliable and efficient models suited for clinical practice. However, current methods suffer from limited anatomical fidelity, restricted axial length, and substantial computational cost, placing them beyond reach for regions with limited resources and infrastructure. We introduce TRACE, a framework that generates 3D medical images with spatiotemporal alignment using a 2D multimodal-conditioned diffusion approach. TRACE models sequential 2D slices as video frame pairs, combining segmentation priors and radiology reports for anatomical alignment, incorporating optical flow to sustain temporal coherence. During inference, an overlapping-frame strategy links frame pairs into a flexible length sequence, reconstructed into a spatiotemporally and anatomically aligned 3D volume. Experimental results demonstrate that TRACE effectively balances computational efficiency with preserving anatomical fidelity and spatiotemporal consistency. Code is available at: https://github.com/VinyehShaw/TRACE.