Patch-based Intuitive Multimodal Prototypes Network (PIMPNet) for Alzheimer's Disease classification

TL;DR

PIMPNet is an interpretable multimodal neural network that combines 3D sMRI data and patient age for Alzheimer's Disease classification, emphasizing interpretability and laying groundwork for future multimodal prototype development.

Contribution

This work introduces PIMPNet, the first interpretable multimodal model integrating 3D neuroimaging and demographic data for AD diagnosis.

Findings

Age prototypes do not enhance predictive accuracy over single modality models.

PIMPNet provides interpretable insights into AD classification.

Foundation established for future multimodal prototype training improvements.

Abstract

Volumetric neuroimaging examinations like structural Magnetic Resonance Imaging (sMRI) are routinely applied to support the clinical diagnosis of dementia like Alzheimer's Disease (AD). Neuroradiologists examine 3D sMRI to detect and monitor abnormalities in brain morphology due to AD, like global and/or local brain atrophy and shape alteration of characteristic structures. There is a strong research interest in developing diagnostic systems based on Deep Learning (DL) models to analyse sMRI for AD. However, anatomical information extracted from an sMRI examination needs to be interpreted together with patient's age to distinguish AD patterns from the regular alteration due to a normal ageing process. In this context, part-prototype neural networks integrate the computational advantages of DL in an interpretable-by-design architecture and showed promising results in medical imaging applications. We present PIMPNet, the first interpretable multimodal model for 3D images and demographics applied to the binary classification of AD from 3D sMRI and patient's age. Despite age prototypes do not improve predictive performance compared to the single modality model, this lays the foundation for future work in the direction of the model's design and multimodal prototype training process