Multimodal Inductive Transfer Learning for Detection of Alzheimer's Dementia and its Severity

TL;DR

This paper introduces a multimodal neural network system that combines acoustic, cognitive, and linguistic features to accurately detect Alzheimer's disease and assess its severity, demonstrating state-of-the-art performance on benchmark datasets.

Contribution

It presents a novel multimodal architecture leveraging transfer learning and specialized neural networks for improved Alzheimer's detection and severity estimation.

Findings

Achieved 83.3% accuracy in AD classification

Attained 4.60 RMSE in MMSE score regression

Surpassed previous benchmarks on DementiaBank datasets

Abstract

Alzheimer's disease is estimated to affect around 50 million people worldwide and is rising rapidly, with a global economic burden of nearly a trillion dollars. This calls for scalable, cost-effective, and robust methods for detection of Alzheimer's dementia (AD). We present a novel architecture that leverages acoustic, cognitive, and linguistic features to form a multimodal ensemble system. It uses specialized artificial neural networks with temporal characteristics to detect AD and its severity, which is reflected through Mini-Mental State Exam (MMSE) scores. We first evaluate it on the ADReSS challenge dataset, which is a subject-independent and balanced dataset matched for age and gender to mitigate biases, and is available through DementiaBank. Our system achieves state-of-the-art test accuracy, precision, recall, and F1-score of 83.3% each for AD classification, and state-of-the-art test root mean squared error (RMSE) of 4.60 for MMSE score regression. To the best of our knowledge, the system further achieves state-of-the-art AD classification accuracy of 88.0% when evaluated on the full benchmark DementiaBank Pitt database. Our work highlights the applicability and transferability of spontaneous speech to produce a robust inductive transfer learning model, and demonstrates generalizability through a task-agnostic feature-space. The source code is available at https://github.com/wazeerzulfikar/alzheimers-dementia