MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion model

TL;DR

This paper introduces MRPD, a novel MRI reconstruction framework that leverages a large latent diffusion model pre-trained on natural images, demonstrating superior generalizability and universality across diverse MRI data scenarios.

Contribution

MRPD is the first method to empirically utilize a large latent diffusion model pre-trained on natural images for universal MRI reconstruction, supporting both database-free and database-available scenarios.

Findings

MRPD outperforms existing methods in out-of-domain MRI reconstruction.

Supports both unsupervised and supervised MRI reconstruction scenarios.

Achieves superior generalizability across different MRI contrasts, organs, and datasets.

Abstract

Implicit visual knowledge in a large latent diffusion model (LLDM) pre-trained on natural images is rich and hypothetically universal to natural and medical images. To test this hypothesis from a practical perspective, we propose a novel framework for undersampled MRI Reconstruction by Prompting a large latent Diffusion model (MRPD). While the existing methods trained on MRI datasets are typically of limited generalizability toward diverse data acquisition scenarios, MRPD supports unsupervised and universally adaptive MRI reconstruction. For unsupervised reconstruction, MRSampler guides LLDM with a random-phase-modulated hard-to-soft control. With any single- or multiple-source MRI dataset, MRPD's performance is boosted universally by a lightweight MRAdapter that only finetunes the LLDM's autoencoder. Experiments on FastMRI and IXI show that MRPD is the only model that supports both MRI database-free and database-available scenarios and attains the best generalizability towards out-of-domain (OOD) samplings, contrasts, and organs among compared unsupervised, supervised, and MRI diffusion methods. To our knowledge, MRPD is the first method that empirically shows the universal prowess of an LLDM pre-trained on vast natural images for MRI. Our official implementation is at https://github.com/Z7Gao/MRPD.