mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis

TL;DR

mustGAN introduces a multi-stream GAN architecture that effectively synthesizes missing MRI contrasts by aggregating information from multiple source images, outperforming previous methods in quality and robustness.

Contribution

This paper presents a novel multi-stream GAN framework that combines multiple one-to-one and a joint many-to-one streams for improved MRI contrast synthesis.

Findings

Superior synthesis quality over state-of-the-art methods

Effective aggregation of multi-source information

Adaptive fusion block enhances performance

Abstract

Multi-contrast MRI protocols increase the level of morphological information available for diagnosis. Yet, the number and quality of contrasts is limited in practice by various factors including scan time and patient motion. Synthesis of missing or corrupted contrasts can alleviate this limitation to improve clinical utility. Common approaches for multi-contrast MRI involve either one-to-one and many-to-one synthesis methods. One-to-one methods take as input a single source contrast, and they learn a latent representation sensitive to unique features of the source. Meanwhile, many-to-one methods receive multiple distinct sources, and they learn a shared latent representation more sensitive to common features across sources. For enhanced image synthesis, here we propose a multi-stream approach that aggregates information across multiple source images via a mixture of multiple one-to-one streams and a joint many-to-one stream. The shared feature maps generated in the many-to-one stream and the complementary feature maps generated in the one-to-one streams are combined with a fusion block. The location of the fusion block is adaptively modified to maximize task-specific performance. Qualitative and quantitative assessments on T1-, T2-, PD-weighted and FLAIR images clearly demonstrate the superior performance of the proposed method compared to previous state-of-the-art one-to-one and many-to-one methods.