# Functional and Clinical: An Explainable Deep Learning Model for Multimodal Alzheimer's Disease Classification

## TL;DR

This paper introduces an explainable deep learning model that combines fMRI and clinical data to accurately classify Alzheimer's disease.

## Contribution

The novel contribution is a multimodal and explainable deep learning model for AD classification with clinical interpretability and reduced data leakage.

## Key findings

- The multimodal model achieved 90% accuracy in classifying Alzheimer's disease from controls.
- Clinical tests like MoCA showed varying importance depending on diagnostic group.
- The model outperformed single-modal fMRI models by a significant margin.

## Abstract

Functional magnetic resonance imaging (fMRI) and deep learning models can classify Alzheimer's disease (AD) with high accuracy. These models are highly adaptable and work with a plethora of architectures, data types, and AD stages. However, fMRI deep learning models lack clinical application due to issues with small datasets, explainability, and reliability (e.g., data leakage).

In this study, we address these issues using multimodal and explainable artificial intelligence (XAI) methods. Specifically, we overcome data size limitations by supplementing fMRI data with clinical tests, use a strict leave‐one‐out cross‐validation regime to control for data leakage, and apply perturbation ranking to explain the importance of features in our model. Our 3D convolutional neural network model was trained and validated on 52 participants from ADNI using five clinical tests and fMRI of the default mode network.

The resulting multimodal model classified AD from controls with an accuracy of 90% and outperformed the same architecture without clinical data (58% accuracy). Our feature rankings showed that clinical tests changed in importance within our model depending on the diagnostic group. For example, our model found the MoCA to be highly important for classifying controls but not for AD. This trend of feature importance was seen across almost all fMRI and clinical features.

Our model was highly accurate and highlighted the importance of combining fMRI and clinical data for AD classification. These findings have implications for the refinement of multimodal deep learning models; however, our small sample and need for external validation are also noted.

A multimodal Alzheimer's classification pipeline that combines clinical tests with fMRI networks to output individual‐level predictions and variable importance metrics.