Temporal Differential Fields for 4D Motion Modeling via Image-to-Video Synthesis

TL;DR

This paper introduces a novel image-to-video synthesis framework with temporal differential fields to accurately model 4D motion in medical imaging, addressing limitations of existing methods that require high-dose scans.

Contribution

It pioneers the use of temporal differential diffusion models and attention mechanisms within an image-to-video synthesis framework for 4D motion modeling in medical imaging.

Findings

Achieves high perceptual similarity and temporal consistency in 4D medical videos.

Effectively models intrinsic motion trajectories in cardiac and lung datasets.

Abstract

Temporal modeling on regular respiration-induced motions is crucial to image-guided clinical applications. Existing methods cannot simulate temporal motions unless high-dose imaging scans including starting and ending frames exist simultaneously. However, in the preoperative data acquisition stage, the slight movement of patients may result in dynamic backgrounds between the first and last frames in a respiratory period. This additional deviation can hardly be removed by image registration, thus affecting the temporal modeling. To address that limitation, we pioneeringly simulate the regular motion process via the image-to-video (I2V) synthesis framework, which animates with the first frame to forecast future frames of a given length. Besides, to promote the temporal consistency of animated videos, we devise the Temporal Differential Diffusion Model to generate temporal differential fields, which measure the relative differential representations between adjacent frames. The prompt attention layer is devised for fine-grained differential fields, and the field augmented layer is adopted to better interact these fields with the I2V framework, promoting more accurate temporal variation of synthesized videos. Extensive results on ACDC cardiac and 4D Lung datasets reveal that our approach simulates 4D videos along the intrinsic motion trajectory, rivaling other competitive methods on perceptual similarity and temporal consistency. Codes will be available soon.