Towards Brain Passage Retrieval -- An Investigation of EEG Query Representations

TL;DR

This paper introduces BPR, a novel brain passage retrieval framework that directly maps EEG signals to relevant text passages, improving retrieval precision without intermediate query translation, and demonstrating promising results on the ZuCo dataset.

Contribution

The paper presents a new EEG-based retrieval method that bypasses query translation, utilizing dense retrieval architectures for direct brain-to-passage matching, with significant performance gains.

Findings

BPR achieves up to 8.81% improvement in precision@5 over EEG-to-text baselines.

Hard negative sampling and specialized brain encoders are crucial for robust cross-modal alignment.

The approach is effective across 30 participants, indicating generalizability.

Abstract

Information Retrieval (IR) systems primarily rely on users' ability to translate their internal information needs into (text) queries. However, this translation process is often uncertain and cognitively demanding, leading to queries that incompletely or inaccurately represent users' true needs. This challenge is particularly acute for users with ill-defined information needs or physical impairments that limit traditional text input, where the gap between cognitive intent and query expression becomes even more pronounced. Recent neuroscientific studies have explored Brain-Machine Interfaces (BMIs) as a potential solution, aiming to bridge the gap between users' cognitive semantics and their search intentions. However, current approaches attempting to decode explicit text queries from brain signals have shown limited effectiveness in learning robust brain-to-text representations, often failing to capture the nuanced semantic information present in brain patterns. To address these limitations, we propose BPR (Brain Passage Retrieval), a novel framework that eliminates the need for intermediate query translation by enabling direct retrieval of relevant passages from users' brain signals. Our approach leverages dense retrieval architectures to map EEG signals and text passages into a shared semantic space. Through comprehensive experiments on the ZuCo dataset, we demonstrate that BPR achieves up to 8.81% improvement in precision@5 over existing EEG-to-text baselines, while maintaining effectiveness across 30 participants. Our ablation studies reveal the critical role of hard negative sampling and specialised brain encoders in achieving robust cross-modal alignment. These results establish the viability of direct brain-to-passage retrieval and provide a foundation for developing more natural interfaces between users' cognitive states and IR systems.