LiFT: Lifted Inter-slice Feature Trajectories for 3D Image Generation from 2D Generators

TL;DR

LiFT introduces a novel framework for 3D medical image synthesis that models inter-slice feature trajectories, achieving high-quality results with significantly reduced computational cost.

Contribution

LiFT factorizes 3D volume generation into per-slice image synthesis and inter-slice trajectory learning, enabling efficient and coherent 3D medical image generation.

Findings

LiFT preserves per-slice quality across datasets.

Approaches the quality of full volumetric models at ~135x lower inference cost.

Improves through-plane coherence in MR-to-CT translation.

Abstract

High-resolution 3D medical image generation remains challenging because fully volumetric models are computationally expensive, while efficient 2D slice generators often fail to preserve anatomical consistency across the third dimension. We propose LiFT, a framework for Lifted inter-slice Feature Trajectories that factorizes 3D volume synthesis into per-slice image generation and inter-slice trajectory learning. Rather than modeling the volumetric distribution end-to-end, LiFT treats a volume as an ordered trajectory in feature space, capturing how anatomical structures appear, transform, and disappear across depth. A tri-planar drifting loss aligns the trajectory of generated slices with the trajectories of real volumes, enabling distributional learning over inter-slice progressions in unconditional generation; in paired translation, a bidirectional $z$-context mixer trained against the registered target supplies through-plane coherence while preserving per-slice fidelity. We evaluate LiFT on BraTS 2023 (unconditional and missing-modality MR) and SynthRAD2023 (MR-to-CT). Across these settings, LiFT preserves per-slice quality, approaches the reported cWDM missing-MR reconstruction quality at $\sim$$135\times$ lower inference cost (without formal equivalence testing), and improves through-plane coherence on MR-to-CT relative to a no-mapper ablation, demonstrating that lightweight inter-slice trajectory learning is a viable route to high-resolution 3D medical synthesis.