MANGO: Multimodal Acuity traNsformer for intelliGent ICU Outcomes

TL;DR

MANGO is a multimodal transformer model that integrates diverse ICU data sources to improve patient acuity prediction, including status transitions and therapy needs, surpassing prior unimodal approaches.

Contribution

This study introduces MANGO, a novel multimodal transformer architecture that fuses EHR, sensor, video, and ambient data for enhanced ICU patient acuity prediction.

Findings

Achieved AUROC of 0.76 for acuity transitions and therapy prediction.

Significantly outperformed models using single modalities.

Demonstrated robustness even with missing modalities.

Abstract

Estimation of patient acuity in the Intensive Care Unit (ICU) is vital to ensure timely and appropriate interventions. Advances in artificial intelligence (AI) technologies have significantly improved the accuracy of acuity predictions. However, prior studies using machine learning for acuity prediction have predominantly relied on electronic health records (EHR) data, often overlooking other critical aspects of ICU stay, such as patient mobility, environmental factors, and facial cues indicating pain or agitation. To address this gap, we present MANGO: the Multimodal Acuity traNsformer for intelliGent ICU Outcomes, designed to enhance the prediction of patient acuity states, transitions, and the need for life-sustaining therapy. We collected a multimodal dataset ICU-Multimodal, incorporating four key modalities, EHR data, wearable sensor data, video of patient's facial cues, and ambient sensor data, which we utilized to train MANGO. The MANGO model employs a multimodal feature fusion network powered by Transformer masked self-attention method, enabling it to capture and learn complex interactions across these diverse data modalities even when some modalities are absent. Our results demonstrated that integrating multiple modalities significantly improved the model's ability to predict acuity status, transitions, and the need for life-sustaining therapy. The best-performing models achieved an area under the receiver operating characteristic curve (AUROC) of 0.76 (95% CI: 0.72-0.79) for predicting transitions in acuity status and the need for life-sustaining therapy, while 0.82 (95% CI: 0.69-0.89) for acuity status prediction...