Shrinking the Teacher: An Adaptive Teaching Paradigm for Asymmetric EEG-Vision Alignment

TL;DR

This paper introduces an adaptive teaching paradigm for asymmetric EEG-vision alignment, where the high-fidelity visual modality dynamically shrinks to better match the noisy EEG signals, improving zero-shot brain-to-image retrieval accuracy.

Contribution

The paper proposes a novel paradigm and a module that enable the vision modality to adaptively shrink, addressing the asymmetry in EEG-vision alignment and enhancing cross-modal retrieval performance.

Findings

Achieved 60.2% top-1 accuracy on zero-shot brain-to-image retrieval

Surpassed previous state-of-the-art by 9.8%

Validated the effectiveness of the adaptive shrinking approach

Abstract

Decoding visual features from EEG signals is a central challenge in neuroscience, with cross-modal alignment as the dominant approach. We argue that the relationship between visual and brain modalities is fundamentally asymmetric, characterized by two critical gaps: a Fidelity Gap (stemming from EEG's inherent noise and signal degradation, vs. vision's high-fidelity features) and a Semantic Gap (arising from EEG's shallow conceptual representation, vs. vision's rich semantic depth). Previous methods often overlook this asymmetry, forcing alignment between the two modalities as if they were equal partners and thereby leading to poor generalization. To address this, we propose the adaptive teaching paradigm. This paradigm empowers the ``teacher" modality (vision) to dynamically shrink and adjust its knowledge structure under task guidance, tailoring its semantically dense features to match the ``student" modality (EEG)'s capacity. We implement this paradigm with the ShrinkAdapter, a simple yet effective module featuring a residual-free design and a bottleneck structure. Through extensive experiments, we validate the underlying rationale and effectiveness of our paradigm. Our method achieves a top-1 accuracy of 60.2\% on the zero-shot brain-to-image retrieval task, surpassing previous state-of-the-art methods by a margin of 9.8\%. Our work introduces a new perspective for asymmetric alignment: the teacher must shrink and adapt to bridge the vision-brain gap.