VM-DDPM: Vision Mamba Diffusion for Medical Image Synthesis

TL;DR

VM-DDPM introduces a novel hybrid architecture combining CNN and State Space Model for medical image synthesis, achieving state-of-the-art results with linear complexity and improved structural texture consistency.

Contribution

The paper presents the first medical image synthesis model based on a hybrid SSM-CNN architecture, integrating multi-level feature extraction and a plug-and-play sequence regeneration strategy.

Findings

Achieves state-of-the-art performance on multiple datasets

Maintains linear computational complexity

Qualitative evaluation by radiologists confirms quality

Abstract

In the realm of smart healthcare, researchers enhance the scale and diversity of medical datasets through medical image synthesis. However, existing methods are limited by CNN local perception and Transformer quadratic complexity, making it difficult to balance structural texture consistency. To this end, we propose the Vision Mamba DDPM (VM-DDPM) based on State Space Model (SSM), fully combining CNN local perception and SSM global modeling capabilities, while maintaining linear computational complexity. Specifically, we designed a multi-level feature extraction module called Multi-level State Space Block (MSSBlock), and a basic unit of encoder-decoder structure called State Space Layer (SSLayer) for medical pathological images. Besides, we designed a simple, Plug-and-Play, zero-parameter Sequence Regeneration strategy for the Cross-Scan Module (CSM), which enabled the S6 module to fully perceive the spatial features of the 2D image and stimulate the generalization potential of the model. To our best knowledge, this is the first medical image synthesis model based on the SSM-CNN hybrid architecture. Our experimental evaluation on three datasets of different scales, i.e., ACDC, BraTS2018, and ChestXRay, as well as qualitative evaluation by radiologists, demonstrate that VM-DDPM achieves state-of-the-art performance.