Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis

TL;DR

This paper introduces BMGAN, a novel 3D bidirectional GAN framework for synthesizing realistic PET images from brain MR scans, effectively capturing semantic and structural information.

Contribution

The paper proposes a bidirectional mapping mechanism within a 3D GAN architecture for improved MR-to-PET synthesis, incorporating dense connections and extensive loss functions.

Findings

Outperforms existing methods in quantitative metrics

Produces perceptually realistic and structurally diverse PET images

Enhances brain structure preservation in synthetic images

Abstract

Fusing multi-modality medical images, such as MR and PET, can provide various anatomical or functional information about human body. But PET data is always unavailable due to different reasons such as cost, radiation, or other limitations. In this paper, we propose a 3D end-to-end synthesis network, called Bidirectional Mapping Generative Adversarial Networks (BMGAN), where image contexts and latent vector are effectively used and jointly optimized for brain MR-to-PET synthesis. Concretely, a bidirectional mapping mechanism is designed to embed the semantic information of PET images into the high dimensional latent space. And the 3D DenseU-Net generator architecture and the extensive objective functions are further utilized to improve the visual quality of synthetic results. The most appealing part is that the proposed method can synthesize the perceptually realistic PET images while preserving the diverse brain structures of different subjects. Experimental results demonstrate that the performance of the proposed method outperforms other competitive cross-modality synthesis methods in terms of quantitative measures, qualitative displays, and classification evaluation.