Incomplete Multimodal Learning for Complex Brain Disorders Prediction

TL;DR

This paper introduces an innovative machine learning approach that leverages auxiliary multimodal data during training to enhance disease prediction accuracy using only neuroimaging data at inference, addressing real-world clinical data limitations.

Contribution

The paper presents a novel incomplete multimodal learning method employing transformers and GANs to improve prediction performance with partial data modalities.

Findings

Outperforms existing methods significantly in predicting cognitive decline.

Effectively utilizes auxiliary data during training to enhance unimodal inference.

Validated on Alzheimer's Disease Neuroimaging Initiative data.

Abstract

Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches typically need a complete set of biomedical data modalities, which may not always be feasible, as some modalities are only available in large-scale research cohorts and are prohibitive to collect in routine clinical practice. Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages. As a result, it is desired to design machine learning models which can use all available data (different data could provide complementary information) during training but conduct inference using only the most common data modality. We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks to effectively exploit auxiliary modalities available during training in order to improve the performance of a unimodal model at inference. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results demonstrate that our approach outperforms the related machine learning and deep learning methods by a significant margin.