EEG classification for visual brain decoding with spatio-temporal and transformer based paradigms

TL;DR

This paper explores EEG classification for visual brain decoding using CNN-BiLSTM and CNN-Transformer models, incorporating feature extraction, window-based classification, and visualization techniques to improve understanding and performance.

Contribution

It introduces a dual-framework approach combining CNN-BiLSTM and CNN-Transformer architectures with specialized feature extraction for EEG classification.

Findings

Both models outperform existing methods on EEG-Imagenet dataset.

Visualization reveals distinct neural signatures for different visual categories.

The CNN-Transformer model demonstrates versatile attention-based learning capabilities.

Abstract

In this work, we delve into the EEG classification task in the domain of visual brain decoding via two frameworks, involving two different learning paradigms. Considering the spatio-temporal nature of EEG data, one of our frameworks is based on a CNN-BiLSTM model. The other involves a CNN-Transformer architecture which inherently involves the more versatile attention based learning paradigm. In both cases, a special 1D-CNN feature extraction module is used to generate the initial embeddings with 1D convolutions in the time and the EEG channel domains. Considering the EEG signals are noisy, non stationary and the discriminative features are even less clear (than in semantically structured data such as text or image), we also follow a window-based classification followed by majority voting during inference, to yield labels at a signal level. To illustrate how brain patterns correlate with different image classes, we visualize t-SNE plots of the BiLSTM embeddings alongside brain activation maps for the top 10 classes. These visualizations provide insightful revelations into the distinct neural signatures associated with each visual category, showcasing the BiLSTM's capability to capture and represent the discriminative brain activity linked to visual stimuli. We demonstrate the performance of our approach on the updated EEG-Imagenet dataset with positive comparisons with state-of-the-art methods.