Longitudinal Ensemble Integration for sequential classification with multimodal data

TL;DR

This paper introduces Longitudinal Ensemble Integration (LEI), a novel framework for multimodal sequential classification, demonstrating superior performance in early dementia detection by effectively integrating diverse longitudinal data over time.

Contribution

The paper presents a new multimodal and longitudinal learning framework, LEI, which improves sequential classification by leveraging intermediate predictions from different data modalities.

Findings

LEI outperforms existing approaches in early dementia detection.

LEI effectively integrates multimodal data over time.

Identifies consistently important features for diagnosis.

Abstract

Effectively modeling multimodal longitudinal data is a pressing need in various application areas, especially biomedicine. Despite this, few approaches exist in the literature for this problem, with most not adequately taking into account the multimodality of the data. In this study, we developed multiple configurations of a novel multimodal and longitudinal learning framework, Longitudinal Ensemble Integration (LEI), for sequential classification. We evaluated LEI's performance, and compared it against existing approaches, for the early detection of dementia, which is among the most studied multimodal sequential classification tasks. LEI outperformed these approaches due to its use of intermediate base predictions arising from the individual data modalities, which enabled their better integration over time. LEI's design also enabled the identification of features that were consistently important across time for the effective prediction of dementia-related diagnoses. Overall, our work demonstrates the potential of LEI for sequential classification from longitudinal multimodal data.