Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity

TL;DR

This paper introduces Inter-Slice Consistent Stochasticity (ISCS), a simple method to improve 3D medical image reconstruction by controlling stochastic noise during diffusion sampling, enhancing inter-slice consistency without extra training or hyper-parameters.

Contribution

The paper proposes ISCS, a plug-and-play strategy that aligns stochastic noise trajectories in diffusion sampling, significantly improving 3D medical imaging quality from 2D diffusion models.

Findings

ISCS improves 3D reconstruction quality in medical imaging.

The method enhances inter-slice consistency without additional training.

It is computationally efficient and easy to integrate into existing pipelines.

Abstract

3D medical imaging is in high demand and essential for clinical diagnosis and scientific research. Currently, diffusion models (DMs) have become an effective tool for medical imaging reconstruction thanks to their ability to learn rich, high-quality data priors. However, learning the 3D data distribution with DMs in medical imaging is challenging, not only due to the difficulties in data collection but also because of the significant computational burden during model training. A common compromise is to train the DMs on 2D data priors and reconstruct stacked 2D slices to address 3D medical inverse problems. However, the intrinsic randomness of diffusion sampling causes severe inter-slice discontinuities of reconstructed 3D volumes. Existing methods often enforce continuity regularizations along the z-axis, which introduces sensitive hyper-parameters and may lead to over-smoothing results. In this work, we revisit the origin of stochasticity in diffusion sampling and introduce Inter-Slice Consistent Stochasticity (ISCS), a simple yet effective strategy that encourages interslice consistency during diffusion sampling. Our key idea is to control the consistency of stochastic noise components during diffusion sampling, thereby aligning their sampling trajectories without adding any new loss terms or optimization steps. Importantly, the proposed ISCS is plug-and-play and can be dropped into any 2D trained diffusion based 3D reconstruction pipeline without additional computational cost. Experiments on several medical imaging problems show that our method can effectively improve the performance of medical 3D imaging problems based on 2D diffusion models. Our findings suggest that controlling inter-slice stochasticity is a principled and practically attractive route toward high-fidelity 3D medical imaging with 2D diffusion priors. The code is available at: https://github.com/duchenhe/ISCS