MM2CT: MR-to-CT translation for multi-modal image fusion with mamba

TL;DR

This paper introduces MM2CT, a novel multi-modal MR-to-CT translation framework utilizing Mamba architecture to effectively fuse T1- and T2-weighted MRI data, improving synthesis quality while reducing computational complexity.

Contribution

The paper presents a new multi-modal MR-to-CT translation method using Mamba, overcoming CNN and Transformer limitations, with modules enhancing MRI-to-CT synthesis performance.

Findings

Achieves state-of-the-art SSIM and PSNR on pelvis dataset

Effectively fuses multi-modal MRI data for improved translation

Demonstrates Mamba's efficiency in medical image synthesis

Abstract

Magnetic resonance (MR)-to-computed tomography (CT) translation offers significant advantages, including the elimination of radiation exposure associated with CT scans and the mitigation of imaging artifacts caused by patient motion. The existing approaches are based on single-modality MR-to-CT translation, with limited research exploring multimodal fusion. To address this limitation, we introduce Multi-modal MR to CT (MM2CT) translation method by leveraging multimodal T1- and T2-weighted MRI data, an innovative Mamba-based framework for multi-modal medical image synthesis. Mamba effectively overcomes the limited local receptive field in CNNs and the high computational complexity issues in Transformers. MM2CT leverages this advantage to maintain long-range dependencies modeling capabilities while achieving multi-modal MR feature integration. Additionally, we incorporate a dynamic local convolution module and a dynamic enhancement module to improve MRI-to-CT synthesis. The experiments on a public pelvis dataset demonstrate that MM2CT achieves state-of-the-art performance in terms of Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR). Our code is publicly available at https://github.com/Gots-ch/MM2CT.