TriFormer: A Multi-modal Transformer Framework For Mild Cognitive Impairment Conversion Prediction

TL;DR

TriFormer is a novel multi-modal transformer framework that integrates imaging and clinical data to improve prediction of MCI conversion to Alzheimer's, demonstrating superior performance on benchmark datasets.

Contribution

The paper introduces TriFormer, a transformer-based model with three specialized transformers for multi-modal data fusion in MCI conversion prediction.

Findings

Outperforms previous state-of-the-art methods.

Effective multi-modal data integration.

Validated on ADNI datasets.

Abstract

The prediction of mild cognitive impairment (MCI) conversion to Alzheimer's disease (AD) is important for early treatment to prevent or slow the progression of AD. To accurately predict the MCI conversion to stable MCI or progressive MCI, we propose Triformer, a novel transformer-based framework with three specialized transformers to incorporate multi-model data. Triformer uses I) an image transformer to extract multi-view image features from medical scans, II) a clinical transformer to embed and correlate multi-modal clinical data, and III) a modality fusion transformer that produces an accurate prediction based on fusing the outputs from the image and clinical transformers. Triformer is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ANDI)1 and ADNI2 datasets and outperforms previous state-of-the-art single and multi-modal methods.