Predicting Brain Degeneration with a Multimodal Siamese Neural Network

TL;DR

This paper introduces a multimodal Siamese neural network that integrates imaging and clinical data from multiple time points to predict neurodegenerative disease progression, demonstrating high accuracy and robustness to missing data.

Contribution

The work presents a novel neural network architecture capable of multimodal learning and handling missing data for neurodegeneration prediction.

Findings

Achieved 92.5% accuracy and 0.978 AUC on test data.

Outperforms unimodal models with up to 37.5% missing clinical data.

Demonstrates robustness to incomplete multimodal data.

Abstract

To study neurodegenerative diseases, longitudinal studies are carried on volunteer patients. During a time span of several months to several years, they go through regular medical visits to acquire data from different modalities, such as biological samples, cognitive tests, structural and functional imaging. These variables are heterogeneous but they all depend on the patient's health condition, meaning that there are possibly unknown relationships between all modalities. Some information may be specific to some modalities, others may be complementary, and others may be redundant. Some data may also be missing. In this work we present a neural network architecture for multimodal learning, able to use imaging and clinical data from two time points to predict the evolution of a neurodegenerative disease, and robust to missing values. Our multimodal network achieves 92.5\% accuracy and an AUC score of 0.978 over a test set of 57 subjects. We also show the superiority of the multimodal architecture, for up to 37.5\% of missing values in test set subjects' clinical measurements, compared to a model using only the clinical modality.