A large-scale evaluation framework for EEG deep learning architectures

TL;DR

This paper introduces a comprehensive evaluation framework for EEG deep learning architectures, enabling systematic comparison across multiple datasets and models to advance BCI applications.

Contribution

It presents a novel, transparent, and reproducible framework for large-scale evaluation of EEG decoding deep learning models across diverse datasets.

Findings

Comparison of CNN architectures on multiple EEG datasets

Framework facilitates analysis of model performance across tasks

Supports reproducibility and transparency in EEG deep learning research

Abstract

EEG is the most common signal source for noninvasive BCI applications. For such applications, the EEG signal needs to be decoded and translated into appropriate actions. A recently emerging EEG decoding approach is deep learning with Convolutional or Recurrent Neural Networks (CNNs, RNNs) with many different architectures already published. Here we present a novel framework for the large-scale evaluation of different deep-learning architectures on different EEG datasets. This framework comprises (i) a collection of EEG datasets currently including 100 examples (recording sessions) from six different classification problems, (ii) a collection of different EEG decoding algorithms, and (iii) a wrapper linking the decoders to the data as well as handling structured documentation of all settings and (hyper-) parameters and statistics, designed to ensure transparency and reproducibility. As an applications example we used our framework by comparing three publicly available CNN architectures: the Braindecode Deep4 ConvNet, Braindecode Shallow ConvNet, and two versions of EEGNet. We also show how our framework can be used to study similarities and differences in the performance of different decoding methods across tasks. We argue that the deep learning EEG framework as described here could help to tap the full potential of deep learning for BCI applications.