Benchmarking ERP Analysis: Manual Features, Deep Learning, and Foundation Models

TL;DR

This paper presents a comprehensive benchmark comparing manual features, deep learning, and foundation models for ERP analysis across multiple datasets, providing guidance for future method selection.

Contribution

It systematically evaluates various approaches for ERP analysis, establishing a unified pipeline and exploring embedding strategies within Transformer models.

Findings

Deep learning models outperform manual features in ERP classification.

Pre-trained EEG foundation models show promising results on ERP tasks.

Embedding strategies significantly impact Transformer model performance on ERP data.

Abstract

Event-related potential (ERP), a specialized paradigm of electroencephalographic (EEG), reflects neurological responses to external stimuli or events, generally associated with the brain's processing of specific cognitive tasks. ERP plays a critical role in cognitive analysis, the detection of neurological diseases, and the assessment of psychological states. Recent years have seen substantial advances in deep learning-based methods for spontaneous EEG and other non-time-locked task-related EEG signals. However, their effectiveness on ERP data remains underexplored, and many existing ERP studies still rely heavily on manually extracted features. In this paper, we conduct a comprehensive benchmark study that systematically compares traditional manual features (followed by a linear classifier), deep learning models, and pre-trained EEG foundation models for ERP analysis. We establish a unified data preprocessing and training pipeline and evaluate these approaches on two representative tasks, ERP stimulus classification and ERP-based brain disease detection, across 12 publicly available datasets. Furthermore, we investigate various token-embedding strategies within advanced Transformer architectures to identify embedding designs that better suit ERP data. Our study provides a landmark framework to guide method selection and tailored model design for future ERP analysis. The code is available at https://github.com/DL4mHealth/ERP-Benchmark