ListenNet: A Lightweight Spatio-Temporal Enhancement Nested Network for Auditory Attention Detection

TL;DR

ListenNet is a novel lightweight neural network that effectively captures spatio-temporal dependencies in EEG signals for improved auditory attention detection, outperforming existing methods while being more efficient.

Contribution

The paper introduces ListenNet, a new neural network architecture with specialized modules for enhanced spatio-temporal feature extraction in EEG-based auditory attention detection.

Findings

Outperforms state-of-the-art methods on three public datasets.

Reduces trainable parameters by approximately 7 times.

Effective in both subject-dependent and subject-independent scenarios.

Abstract

Auditory attention detection (AAD) aims to identify the direction of the attended speaker in multi-speaker environments from brain signals, such as Electroencephalography (EEG) signals. However, existing EEG-based AAD methods overlook the spatio-temporal dependencies of EEG signals, limiting their decoding and generalization abilities. To address these issues, this paper proposes a Lightweight Spatio-Temporal Enhancement Nested Network (ListenNet) for AAD. The ListenNet has three key components: Spatio-temporal Dependency Encoder (STDE), Multi-scale Temporal Enhancement (MSTE), and Cross-Nested Attention (CNA). The STDE reconstructs dependencies between consecutive time windows across channels, improving the robustness of dynamic pattern extraction. The MSTE captures temporal features at multiple scales to represent both fine-grained and long-range temporal patterns. In addition, the CNA integrates hierarchical features more effectively through novel dynamic attention mechanisms to capture deep spatio-temporal correlations. Experimental results on three public datasets demonstrate the superiority of ListenNet over state-of-the-art methods in both subject-dependent and challenging subject-independent settings, while reducing the trainable parameter count by approximately 7 times. Code is available at:https://github.com/fchest/ListenNet.