Digital FAST: An AI-Driven Multimodal Framework for Rapid and Early Stroke Screening

TL;DR

This paper introduces a multimodal deep learning framework that combines facial, speech, and movement data for rapid, non-invasive stroke screening, achieving high accuracy and robustness in early detection.

Contribution

It presents a novel AI-driven multimodal approach integrating facial, speech, and movement data with attention-based fusion for early stroke detection.

Findings

Achieved 95.83% accuracy and 96.00% F1-score on a self-collected dataset.

Outperformed unimodal models, demonstrating the effectiveness of multimodal integration.

Detected all stroke cases in the test set, showing high sensitivity.

Abstract

Early identification of stroke symptoms is essential for enabling timely intervention and improving patient outcomes, particularly in prehospital settings. This study presents a fast, non-invasive multimodal deep learning framework for automatic binary stroke screening based on data collected during the F.A.S.T. assessment. The proposed approach integrates complementary information from facial expressions, speech signals, and upper-body movements to enhance diagnostic robustness. Facial dynamics are represented using landmark based features and modeled with a Transformer architecture to capture temporal dependencies. Speech signals are converted into mel spectrograms and processed using an Audio Spectrogram Transformer, while upper-body pose sequences are analyzed with an MLP-Mixer network to model spatiotemporal motion patterns. The extracted modality specific representations are combined through an attention-based fusion mechanism to effectively learn cross modal interactions. Experiments conducted on a self-collected dataset of 222 videos from 37 subjects demonstrate that the proposed multimodal model consistently outperforms unimodal baselines, achieving 95.83% accuracy and a 96.00% F1-score. The model attains a strong balance between sensitivity and specificity and successfully detects all stroke cases in the test set. These results highlight the potential of multimodal learning and transfer learning for early stroke screening, while emphasizing the need for larger, clinically representative datasets to support reliable real-world deployment.