Cross-Modality Neuroimage Synthesis: A Survey

TL;DR

This survey reviews methods for synthesizing neuroimages across modalities using unsupervised and weakly supervised learning, addressing challenges, architectures, evaluation, and applications in brain research.

Contribution

It provides a comprehensive overview of cross-modality neuroimage synthesis techniques, highlighting recent advances and future research directions.

Findings

Analyzes various architectures for neuroimage synthesis.

Evaluates how synthesis improves downstream tasks.

Summarizes datasets and evaluation metrics.

Abstract

Multi-modality imaging improves disease diagnosis and reveals distinct deviations in tissues with anatomical properties. The existence of completely aligned and paired multi-modality neuroimaging data has proved its effectiveness in brain research. However, collecting fully aligned and paired data is expensive or even impractical, since it faces many difficulties, including high cost, long acquisition time, image corruption, and privacy issues. An alternative solution is to explore unsupervised or weakly supervised learning methods to synthesize the absent neuroimaging data. In this paper, we provide a comprehensive review of cross-modality synthesis for neuroimages, from the perspectives of weakly supervised and unsupervised settings, loss functions, evaluation metrics, imaging modalities, datasets, and downstream applications based on synthesis. We begin by highlighting several opening challenges for cross-modality neuroimage synthesis. Then, we discuss representative architectures of cross-modality synthesis methods under different supervisions. This is followed by a stepwise in-depth analysis to evaluate how cross-modality neuroimage synthesis improves the performance of its downstream tasks. Finally, we summarize the existing research findings and point out future research directions. All resources are available at https://github.com/M-3LAB/awesome-multimodal-brain-image-systhesis