Generative Drifting for Conditional Medical Image Generation

TL;DR

GDM is a novel framework for 3D medical image generation that balances plausibility, fidelity, and efficiency through a multi-objective drifting approach, outperforming existing methods in MRI-to-CT and CT reconstruction tasks.

Contribution

The paper introduces GDM, a drifting-based generative framework that extends to 3D medical imaging with multi-level features and gradient coordination for improved quality and efficiency.

Findings

GDM outperforms GAN, flow-matching, SDE, and supervised methods in experiments.

GDM achieves better balance among fidelity, realism, and inference speed.

GDM is effective for MRI-to-CT synthesis and sparse-view CT reconstruction.

Abstract

Conditional medical image generation plays an important role in many clinically relevant imaging tasks. However, existing methods still face a fundamental challenge in balancing inference efficiency, patient-specific fidelity, and distribution-level plausibility, particularly in high-dimensional 3D medical imaging. In this work, we propose GDM, a generative drifting framework that reformulates deterministic medical image prediction as a multi-objective learning problem to jointly promote distribution-level plausibility and patient-specific fidelity while retaining one-step inference. GDM extends drifting to 3D medical imaging through an attractive-repulsive drift that minimizes the discrepancy between the generator pushforward and the target distribution. To enable stable drifting-based learning in 3D volumetric data, GDM constructs a multi-level feature bank from a medical foundation encoder to support reliable affinity estimation and drifting field computation across complementary global, local, and spatial representations. In addition, a gradient coordination strategy in the shared output space improves optimization balance under competing distribution-level and fidelity-oriented objectives. We evaluate the proposed framework on two representative tasks, MRI-to-CT synthesis and sparse-view CT reconstruction. Experimental results show that GDM consistently outperforms a wide range of baselines, including GAN-based, flow-matching-based, and SDE-based generative models, as well as supervised regression methods, while improving the balance among anatomical fidelity, quantitative reliability, perceptual realism, and inference efficiency. These findings suggest that GDM provides a practical and effective framework for conditional 3D medical image generation.