# A multimodal deep learning model for predicting early neurological deterioration in patients with acute ischemic stroke

## TL;DR

A deep learning model combining clinical data and imaging reports improves prediction of neurological decline in stroke patients.

## Contribution

A novel multimodal deep learning model that integrates structured clinical data and radiology text for END prediction in acute ischemic stroke.

## Key findings

- The multimodal model achieved AUCs of 0.877 (training) and 0.771 (test) for predicting END.
- Key predictors included D-dimer, diastolic blood pressure, and textual descriptors like 'unsteadiness' and 'vision'.
- The model effectively stratified patients into high- and low-risk groups with significantly different END outcomes.

## Abstract

Timely identification of patients at high risk for early neurological deterioration (END) is critical for effective intervention after acute ischemic stroke; however, existing prediction models largely rely on structured clinical data and underutilize semantic information from imaging findings.

In this retrospective single-center study at Xiangyang No.1 People’s Hospital (January 2018–December 2023), 426 patients with acute ischemic stroke and imaging-confirmed middle cerebral artery occlusion who received non-endovascular treatment were included. Patients with other arterial occlusions, endovascular therapy, or incomplete data were excluded. END was defined as a ≥2-point increase in total National Institutes of Health Stroke Scale (NIHSS) score or a ≥1-point increase in the motor subscore within 7 days after admission. Structured clinical variables and radiology report text collected at admission were integrated into a multimodal deep learning model. Model performance was evaluated using AUC, accuracy, recall, precision, F1-score, calibration, and decision curve analysis, with interpretability assessed using SHAP and Integrated Gradients.

Among all patients, 38.0% exhibited END (30.3% early; 7.7% late). The multimodal Concat-Fusion model achieved AUCs of 0.877 (training) and 0.771 (test), surpassing single-modality models, with strong predictive capabilities for early (AUC = 0.842) and late END (AUC = 0.855). Subgroup analyses confirmed consistent performance across NIHSS scores, D-dimer levels, hypertension, and atrial fibrillation, with significantly higher AUC in diabetic patients (p = 0.002). SHAP analysis identified D-dimer, diastolic blood pressure, and heart rate as key contributors. Integrated Gradients revealed symptom descriptors including “unsteadiness,” “walking,” and “vision” as significant textual predictors. Risk stratification effectively distinguished high- and low-risk groups with significantly different cumulative END incidences within 48 h and 7 days (log-rank p < 0.0001). High-risk patients exhibited poorer NIHSS trajectories during 14-day follow-up.

This multimodal prediction model may improve early identification of END and support individualized clinical management. Larger prospective studies are required to validate its clinical utility.