Human brain activity for machine attention
Lukas Muttenthaler, Nora Hollenstein, Maria Barrett

TL;DR
This paper introduces a novel method to incorporate EEG brain activity data into neural attention models for NLP, demonstrating improved performance in relation classification tasks by leveraging neuroscientific signals.
Contribution
It is the first to utilize EEG data to inform and regularize neural attention mechanisms in NLP models, bridging neuroscience and machine learning.
Findings
EEG features can distinguish reading tasks after pre-processing.
Incorporating EEG improves relation classification accuracy.
Task difficulty and EEG frequency influence the benefit.
Abstract
Cognitively inspired NLP leverages human-derived data to teach machines about language processing mechanisms. Recently, neural networks have been augmented with behavioral data to solve a range of NLP tasks spanning syntax and semantics. We are the first to exploit neuroscientific data, namely electroencephalography (EEG), to inform a neural attention model about language processing of the human brain. The challenge in working with EEG data is that features are exceptionally rich and need extensive pre-processing to isolate signals specific to text processing. We devise a method for finding such EEG features to supervise machine attention through combining theoretically motivated cropping with random forest tree splits. After this dimensionality reduction, the pre-processed EEG features are capable of distinguishing two reading tasks retrieved from a publicly available EEG corpus. We…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEEG and Brain-Computer Interfaces · Neural Networks and Applications · Topic Modeling
