A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

TL;DR

This paper introduces EEGVFusion, a multimodal neural network that combines EEG and video data with self-supervised learning and alignment techniques to improve seizure detection accuracy in mouse models.

Contribution

The work presents a novel multimodal framework integrating EEG and video analysis with a curated dataset, achieving high accuracy and low false alarms in seizure detection.

Findings

Achieved 99.57% balanced accuracy with perfect event sensitivity.

Reduced false alarm rate from 2.725 to 0.4833 FP/h in single-subject evaluation.

EEG pre-training and optimal-transport alignment improve detection performance.

Abstract

Reliable seizure detection in mouse models is essential for preclinical epilepsy research, yet manual review of synchronized video-EEG recordings is labor-intensive and single-modality systems fail for complementary reasons: video-based methods are easily confounded by benign behaviors, whereas EEG-based methods are vulnerable to ictal motion artifacts. We present EEGVFusion, a multimodal framework that combines self-supervised EEG representation learning, spatio-temporal video encoding, optimal-transport alignment, and bidirectional cross-attention to integrate neural and behavioral evidence. We also curate an expert-annotated dataset of synchronized EEG and video recordings comprising 93 sessions from 15 mice for training and evaluation. In the random-session split, EEGVFusion achieved a Balanced Accuracy of 0.9957 with perfect event sensitivity and an Event FAR of 0.6250 FP/h, indicating strong seizure detection performance with a low false-alarm burden. In a single held-out-subject evaluation with Subject 110 reserved for testing, EEGVFusion achieved a Balanced Accuracy of 0.9718 and reduced Event FAR from 2.7250 FP/h for the EEG-only counterpart to 0.4833 FP/h while preserving perfect event sensitivity. Targeted ablations further showed that EEG pre-training and OT alignment help reduce false alarms while preserving event sensitivity.