D-FaST: Cognitive Signal Decoding with Disentangled Frequency-Spatial-Temporal Attention

TL;DR

This paper introduces D-FaST, a novel disentangled frequency-spatial-temporal attention framework for cognitive signal decoding, achieving state-of-the-art accuracy on multiple datasets and advancing the integration of multi-domain features in cognitive language processing.

Contribution

The paper proposes a new D-FaST model with disentangled frequency, spatial, and temporal attention mechanisms, and introduces the MNRED dataset for improved cognitive signal decoding.

Findings

D-FaST achieves 78.72% accuracy on MNRED, surpassing previous methods.

D-FaST attains 78.35% accuracy on ZuCo, 74.85% on BCIC IV-2A, and 76.81% on BCIC IV-2B.

The model significantly outperforms existing approaches across multiple datasets.

Abstract

Cognitive Language Processing (CLP), situated at the intersection of Natural Language Processing (NLP) and cognitive science, plays a progressively pivotal role in the domains of artificial intelligence, cognitive intelligence, and brain science. Among the essential areas of investigation in CLP, Cognitive Signal Decoding (CSD) has made remarkable achievements, yet there still exist challenges related to insufficient global dynamic representation capability and deficiencies in multi-domain feature integration. In this paper, we introduce a novel paradigm for CLP referred to as Disentangled Frequency-Spatial-Temporal Attention(D-FaST). Specifically, we present an novel cognitive signal decoder that operates on disentangled frequency-space-time domain attention. This decoder encompasses three key components: frequency domain feature extraction employing multi-view attention, spatial domain feature extraction utilizing dynamic brain connection graph attention, and temporal feature extraction relying on local time sliding window attention. These components are integrated within a novel disentangled framework. Additionally, to encourage advancements in this field, we have created a new CLP dataset, MNRED. Subsequently, we conducted an extensive series of experiments, evaluating D-FaST's performance on MNRED, as well as on publicly available datasets including ZuCo, BCIC IV-2A, and BCIC IV-2B. Our experimental results demonstrate that D-FaST outperforms existing methods significantly on both our datasets and traditional CSD datasets including establishing a state-of-the-art accuracy score 78.72% on MNRED, pushing the accuracy score on ZuCo to 78.35%, accuracy score on BCIC IV-2A to 74.85% and accuracy score on BCIC IV-2B to 76.81%.