SIGNAL: Dataset for Semantic and Inferred Grammar Neurological Analysis of Language

TL;DR

This paper introduces a dataset combining EEG recordings and sentences to study brain and language model alignment.

Contribution

The dataset includes both congruent and incongruent sentences with EEG data, enabling brain-model alignment research.

Findings

The dataset contains 600 sentences with EEG recordings from 21 participants.

Validation confirmed the dataset's suitability for brain-model alignment studies.

Stimuli were assessed by native speakers and used in LLM probing.

Abstract

Recently, the idea of brain-model alignment has been the topic of several influential works. However, most of previous studies were based on datasets collected during regular reading tasks where the subjects were not exposed to processing linguistic incongruencies, and stimuli were not controlled for key linguistic properties. Meanwhile, interpretability studies of Large Language Models pay growing attention to thoroughly designed linguistic tasks based on certain acceptability measures. We present a dataset that contains 600 sentences with a combination of congruent and grammatically or/and semantically incongruent sentences coupled with high density 64-channel EEG recordings of 21 participants. The text stimuli were assessed by native speakers and later used in EEG recording and validation and LLM probing. The validation results proved suitability of the data for future research on brain-model alignment in the linguistic context.

Figures