Integrating Language-Image Prior into EEG Decoding for Cross-Task Zero-Calibration RSVP-BCI

TL;DR

This paper introduces a novel EEG decoding model that integrates language-image priors to improve cross-task zero-calibration RSVP-BCI performance, enabling more practical and adaptable brain-computer interfaces.

Contribution

The study proposes ELIPformer, a transformer-based model with language-image priors and attention mechanisms, for enhanced cross-task EEG decoding without calibration data.

Findings

Achieves superior cross-task decoding accuracy

Effectively fuses EEG with language-image features

Demonstrates practical potential for RSVP-BCI applications

Abstract

Rapid Serial Visual Presentation (RSVP)-based Brain-Computer Interface (BCI) is an effective technology used for information detection by detecting Event-Related Potentials (ERPs). The current RSVP decoding methods can perform well in decoding EEG signals within a single RSVP task, but their decoding performance significantly decreases when directly applied to different RSVP tasks without calibration data from the new tasks. This limits the rapid and efficient deployment of RSVP-BCI systems for detecting different categories of targets in various scenarios. To overcome this limitation, this study aims to enhance the cross-task zero-calibration RSVP decoding performance. First, we design three distinct RSVP tasks for target image retrieval and build an open-source dataset containing EEG signals and corresponding stimulus images. Then we propose an EEG with Language-Image Prior fusion Transformer (ELIPformer) for cross-task zero-calibration RSVP decoding. Specifically, we propose a prompt encoder based on the language-image pre-trained model to extract language-image features from task-specific prompts and stimulus images as prior knowledge for enhancing EEG decoding. A cross bidirectional attention mechanism is also adopted to facilitate the effective feature fusion and alignment between the EEG and language-image features. Extensive experiments demonstrate that the proposed model achieves superior performance in cross-task zero-calibration RSVP decoding, which promotes the RSVP-BCI system from research to practical application.