From Clever Hans to Scientific Discovery: Interpreting EEG Foundational Transformers with LRP

TL;DR

This paper explores the use of Layer-wise Relevance Propagation (LRP) to interpret EEG foundation models based on Transformers, revealing model behaviors and hypotheses in brain-computer interface applications.

Contribution

It extends LRP to Transformer-based EEG models, demonstrating its effectiveness in verifying decisions and uncovering biologically plausible insights.

Findings

LRP can verify EEG-FM decisions and surface hypotheses.

In motor imagery, LRP reveals reliance on ocular signals over motor signals.

In affect prediction, LRP suggests a sensorimotor signature of arousal.

Abstract

Emerging foundation models (FMs) in electroencephalography (EEG) promise a path to scale deep learning in diagnostics and brain-computer interfaces despite data scarcity, yet their opaque nature remains a barrier to wider adoption. We investigate attention-aware Layer-wise relevance propagation (LRP) as a post-hoc attribution method for EEG-FMs, extending LRP's use on convolutional neural network (CNN)-based EEG models to the Transformer architectures that current FMs are based on. We find that LRP can both verify EEG-FM decisions and surface novel, biologically plausible hypotheses from them. In motor imagery, it unmasks 'Clever Hans' behavior where models prioritize task correlated ocular signals over the intended motor correlates. In a naturalistic paradigm for affect prediction, it reveals a recurring reliance on a central electrode cluster, suggesting a candidate sensorimotor signature of arousal. Though heatmap interpretation remains ambiguous in this complex domain, the results position LRP as a tool for both verification and exploration of EEG-FMs, a role that will grow in both importance and discovery potential as the underlying models mature.