S$^2$M-Former: Spiking Symmetric Mixing Branchformer for Brain Auditory Attention Detection

TL;DR

S$^2$M-Former introduces a brain-inspired spiking neural network architecture for auditory attention detection, achieving high accuracy with significantly reduced power consumption and parameters, suitable for neuro-steered hearing devices.

Contribution

The paper presents a novel spiking symmetric mixing framework with lightweight token sequences, enhancing EEG feature learning while reducing energy use and parameters.

Findings

Achieves state-of-the-art accuracy on three AAD benchmarks.

Reduces power consumption by 5.8× compared to recent ANN methods.

Lowers model parameters by 14.7×, improving efficiency.

Abstract

Auditory attention detection (AAD) aims to decode listeners' focus in complex auditory environments from electroencephalography (EEG) recordings, which is crucial for developing neuro-steered hearing devices. Despite recent advancements, EEG-based AAD remains hindered by the absence of synergistic frameworks that can fully leverage complementary EEG features under energy-efficiency constraints. We propose S$^2$M-Former, a novel spiking symmetric mixing framework to address this limitation through two key innovations: i) Presenting a spike-driven symmetric architecture composed of parallel spatial and frequency branches with mirrored modular design, leveraging biologically plausible token-channel mixers to enhance complementary learning across branches; ii) Introducing lightweight 1D token sequences to replace conventional 3D operations, reducing parameters by 14.7$\times$. The brain-inspired spiking architecture further reduces power consumption, achieving a 5.8$\times$ energy reduction compared to recent ANN methods, while also surpassing existing SNN baselines in terms of parameter efficiency and performance. Comprehensive experiments on three AAD benchmarks (KUL, DTU and AV-GC-AAD) across three settings (within-trial, cross-trial and cross-subject) demonstrate that S$^2$M-Former achieves comparable state-of-the-art (SOTA) decoding accuracy, making it a promising low-power, high-performance solution for AAD tasks. Code is available at https://github.com/JackieWang9811/S2M-Former.