Cross-modal Medical Image Generation Based on Pyramid Convolutional Attention Network

TL;DR

This paper presents a novel pyramid convolutional attention network that efficiently generates high-quality PET images from sMRI scans, aiding Alzheimer's diagnosis by filling missing imaging modalities.

Contribution

The proposed model combines pyramid convolution, channel attention, and self-attention mechanisms with new loss functions for improved PET image synthesis from sMRI.

Findings

Generated images outperform previous methods in accuracy and similarity metrics.

The synthesized PET images significantly improve AD diagnosis accuracy.

The method achieves close to real image quality and enhances clinical decision-making.

Abstract

The integration of multimodal medical imaging can provide complementary and comprehensive information for the diagnosis of Alzheimer's disease (AD). However, in clinical practice, since positron emission tomography (PET) is often missing, multimodal images might be incomplete. To address this problem, we propose a method that can efficiently utilize structural magnetic resonance imaging (sMRI) image information to generate high-quality PET images. Our generation model efficiently utilizes pyramid convolution combined with channel attention mechanism to extract multi-scale local features in sMRI, and injects global correlation information into these features using self-attention mechanism to ensure the restoration of the generated PET image on local texture and global structure. Additionally, we introduce additional loss functions to guide the generation model in producing higher-quality PET images. Through experiments conducted on publicly available ADNI databases, the generated images outperform previous research methods in various performance indicators (average absolute error: 0.0194, peak signal-to-noise ratio: 29.65, structural similarity: 0.9486) and are close to real images. In promoting AD diagnosis, the generated images combined with their corresponding sMRI also showed excellent performance in AD diagnosis tasks (classification accuracy: 94.21 %), and outperformed previous research methods of the same type. The experimental results demonstrate that our method outperforms other competing methods in quantitative metrics, qualitative visualization, and evaluation criteria.