Functional connectivity guided deep neural network for decoding high-level visual imagery

TL;DR

This paper presents a novel EEG-based BCI system that uses high-level visual imagery and deep learning to decode user intentions for controlling robotic arms, demonstrating promising real-time and subject-independent performance.

Contribution

Introduces a deep learning framework combining functional connectivity and CNN-Transformer for decoding high-level visual imagery in EEG-based BCI applications.

Findings

Effective decoding of complex visual imagery intentions

Enhanced robotic arm control accuracy

Subject-independent validation through cross-validation

Abstract

This study introduces a pioneering approach in brain-computer interface (BCI) technology, featuring our novel concept of high-level visual imagery for non-invasive electroencephalography (EEG)-based communication. High-level visual imagery, as proposed in our work, involves the user engaging in the mental visualization of complex upper limb movements. This innovative approach significantly enhances the BCI system, facilitating the extension of its applications to more sophisticated tasks such as EEG-based robotic arm control. By leveraging this advanced form of visual imagery, our study opens new horizons for intricate and intuitive mind-controlled interfaces. We developed an advanced deep learning architecture that integrates functional connectivity metrics with a convolutional neural network-image transformer. This framework is adept at decoding subtle user intentions, addressing the spatial variability in high-level visual tasks, and effectively translating these into precise commands for robotic arm control. Our comprehensive offline and pseudo-online evaluations demonstrate the framework's efficacy in real-time applications, including the nuanced control of robotic arms. The robustness of our approach is further validated through leave-one-subject-out cross-validation, marking a significant step towards versatile, subject-independent BCI applications. This research highlights the transformative impact of advanced visual imagery and deep learning in enhancing the usability and adaptability of BCI systems, particularly in robotic arm manipulation.