A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis

TL;DR

This paper introduces a deep learning model that jointly reconstructs and synthesizes multi-modal MRI images from incomplete data, reducing acquisition time while maintaining high image quality.

Contribution

It presents a novel variational deep-learning framework with a learnable optimization algorithm for joint MRI reconstruction and synthesis from incomplete multi-modal data.

Findings

Effective reconstruction and synthesis demonstrated on extensive experiments.

Reduces MRI data acquisition time without compromising image quality.

Outperforms existing methods in multi-modal MRI tasks.

Abstract

Generating multi-contrasts/modal MRI of the same anatomy enriches diagnostic information but is limited in practice due to excessive data acquisition time. In this paper, we propose a novel deep-learning model for joint reconstruction and synthesis of multi-modal MRI using incomplete k-space data of several source modalities as inputs. The output of our model includes reconstructed images of the source modalities and high-quality image synthesized in the target modality. Our proposed model is formulated as a variational problem that leverages several learnable modality-specific feature extractors and a multimodal synthesis module. We propose a learnable optimization algorithm to solve this model, which induces a multi-phase network whose parameters can be trained using multi-modal MRI data. Moreover, a bilevel-optimization framework is employed for robust parameter training. We demonstrate the effectiveness of our approach using extensive numerical experiments.