Multiscale fusion enhanced spiking neural network for invasive BCI neural signal decoding

TL;DR

This paper introduces a multiscale fusion spiking neural network that improves real-time, energy-efficient decoding of invasive BCI signals by mimicking human visual processing and integrating advanced feature extraction techniques.

Contribution

The paper proposes a novel multiscale fusion enhanced SNN architecture that improves decoding accuracy, robustness, and energy efficiency for invasive BCI signal processing.

Findings

Outperforms traditional neural networks like MLP and GRU in accuracy.

Demonstrates robustness in cross-day signal decoding.

Suitable for neuromorphic chip implementation for online decoding.

Abstract

Brain-computer interfaces (BCIs) are an advanced fusion of neuroscience and artificial intelligence, requiring stable and long-term decoding of neural signals. Spiking Neural Networks (SNNs), with their neuronal dynamics and spike-based signal processing, are inherently well-suited for this task. This paper presents a novel approach utilizing a Multiscale Fusion enhanced Spiking Neural Network (MFSNN). The MFSNN emulates the parallel processing and multiscale feature fusion seen in human visual perception to enable real-time, efficient, and energy-conserving neural signal decoding. Initially, the MFSNN employs temporal convolutional networks and channel attention mechanisms to extract spatiotemporal features from raw data. It then enhances decoding performance by integrating these features through skip connections. Additionally, the MFSNN improves generalizability and robustness in cross-day signal decoding through mini-batch supervised generalization learning. In two benchmark invasive BCI paradigms, including the single-hand grasp-and-touch and center-and-out reach tasks, the MFSNN surpasses traditional artificial neural network methods, such as MLP and GRU, in both accuracy and computational efficiency. Moreover, the MFSNN's multiscale feature fusion framework is well-suited for the implementation on neuromorphic chips, offering an energy-efficient solution for online decoding of invasive BCI signals.