Fusing multimodal neuroimaging data with a variational autoencoder

TL;DR

This paper introduces a scalable and interpretable variational autoencoder approach to fuse multiple neuroimaging modalities, demonstrating effective schizophrenia classification with high ROC-AUC scores.

Contribution

It proposes a novel VAE-based method for multimodal neuroimaging data fusion, emphasizing scalability and interpretability.

Findings

Support vector machine on learned representations achieves ROC-AUC of 0.8610.

The method effectively captures shared and exclusive information across modalities.

Initial results suggest promising application in neuropsychiatric disorder classification.

Abstract

Neuroimaging studies often involve the collection of multiple data modalities. These modalities contain both shared and mutually exclusive information about the brain. This work aims at finding a scalable and interpretable method to fuse the information of multiple neuroimaging modalities using a variational autoencoder (VAE). To provide an initial assessment, this work evaluates the representations that are learned using a schizophrenia classification task. A support vector machine trained on the representations achieves an area under the curve for the classifier's receiver operating characteristic (ROC-AUC) of 0.8610.