Alzheimer's Disease Prediction via Brain Structural-Functional Deep Fusing Network

TL;DR

This paper introduces a novel deep learning model, CT-GAN, that fuses structural and functional brain imaging data to improve Alzheimer's disease prediction and identify disease-related neural connections.

Contribution

The paper proposes a cross-modal transformer GAN that effectively fuses multimodal neuroimages and enhances AD prediction accuracy with a new swapping bi-attention mechanism.

Findings

Significantly improves AD prediction performance on ADNI dataset.

Effectively identifies AD-related brain connections.

Provides new insights into neural circuits associated with AD.

Abstract

Fusing structural-functional images of the brain has shown great potential to analyze the deterioration of Alzheimer's disease (AD). However, it is a big challenge to effectively fuse the correlated and complementary information from multimodal neuroimages. In this paper, a novel model termed cross-modal transformer generative adversarial network (CT-GAN) is proposed to effectively fuse the functional and structural information contained in functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI). The CT-GAN can learn topological features and generate multimodal connectivity from multimodal imaging data in an efficient end-to-end manner. Moreover, the swapping bi-attention mechanism is designed to gradually align common features and effectively enhance the complementary features between modalities. By analyzing the generated connectivity features, the proposed model can identify AD-related brain connections. Evaluations on the public ADNI dataset show that the proposed CT-GAN can dramatically improve prediction performance and detect AD-related brain regions effectively. The proposed model also provides new insights for detecting AD-related abnormal neural circuits.