Category-aware EEG image generation based on wavelet transform and contrast semantic loss

TL;DR

This paper introduces a transformer-based EEG-to-image reconstruction model utilizing wavelet transforms and semantic loss, significantly improving semantic alignment and classification accuracy in brain-computer interface applications.

Contribution

It presents a novel EEG encoder with wavelet transform and gating, combined with a diffusion model and semantic loss, advancing EEG-based image reconstruction methods.

Findings

Achieved up to 43% single-subject classification accuracy

Enhanced semantic alignment and image reconstruction quality

Outperformed existing state-of-the-art methods

Abstract

Reconstructing visual stimuli from EEG signals is a crucial step in realizing brain-computer interfaces. In this paper, we propose a transformer-based EEG signal encoder integrating the Discrete Wavelet Transform (DWT) and the gating mechanism. Guided by the feature alignment and category-aware fusion losses, this encoder is used to extract features related to visual stimuli from EEG signals. Subsequently, with the aid of a pre-trained diffusion model, these features are reconstructed into visual stimuli. To verify the effectiveness of the model, we conducted EEG-to-image generation and classification tasks using the THINGS-EEG dataset. To address the limitations of quantitative analysis at the semantic level, we combined WordNet-based classification and semantic similarity metrics to propose a novel semantic-based score, emphasizing the ability of our model to transfer neural activities into visual representations. Experimental results show that our model significantly improves semantic alignment and classification accuracy, which achieves a maximum single-subject accuracy of 43\%, outperforming other state-of-the-art methods. The source code and supplementary material is available at https://github.com/zes0v0inn/DWT_EEG_Reconstruction/tree/main.