Dual Stream Graph Transformer Fusion Networks for Enhanced Brain Decoding

TL;DR

This paper introduces a dual stream graph-transformer architecture for classifying task-based MEG data, combining spatial graph attention networks with temporal transformers to improve brain decoding accuracy.

Contribution

The novel DS-GTF architecture integrates spatial graph attention and temporal transformers for MEG classification, with new adjacency initialization methods.

Findings

Improved classification accuracy over existing models

Reduced standard deviation across test subjects

Effective fusion of spatial and temporal features

Abstract

This paper presents the novel Dual Stream Graph-Transformer Fusion (DS-GTF) architecture designed specifically for classifying task-based Magnetoencephalography (MEG) data. In the spatial stream, inputs are initially represented as graphs, which are then passed through graph attention networks (GAT) to extract spatial patterns. Two methods, TopK and Thresholded Adjacency are introduced for initializing the adjacency matrix used in the GAT. In the temporal stream, the Transformer Encoder receives concatenated windowed input MEG data and learns new temporal representations. The learned temporal and spatial representations from both streams are fused before reaching the output layer. Experimental results demonstrate an enhancement in classification performance and a reduction in standard deviation across multiple test subjects compared to other examined models.