Multimodal Brain-Computer Interfaces: AI-powered Decoding Methodologies

TL;DR

This review discusses recent AI-driven algorithms for multimodal brain-computer interfaces, focusing on decoding techniques, emerging architectures, and challenges in integrating diverse brain data modalities.

Contribution

It provides a comprehensive analysis of current decoding algorithms and highlights future directions like multimodal Transformers in BCI research.

Findings

AI algorithms improve multimodal BCI decoding accuracy

Emerging architectures like Transformers show promise for data integration

Challenges include data heterogeneity and error mitigation

Abstract

Brain-computer interfaces (BCIs) enable direct communication between the brain and external devices. This review highlights the core decoding algorithms that enable multimodal BCIs, including a dissection of the elements, a unified view of diversified approaches, and a comprehensive analysis of the present state of the field. We emphasize algorithmic advancements in cross-modality mapping, sequential modeling, besides classic multi-modality fusion, illustrating how these novel AI approaches enhance decoding of brain data. The current literature of BCI applications on visual, speech, and affective decoding are comprehensively explored. Looking forward, we draw attention on the impact of emerging architectures like multimodal Transformers, and discuss challenges such as brain data heterogeneity and common errors. This review also serves as a bridge in this interdisciplinary field for experts with neuroscience background and experts that study AI, aiming to provide a comprehensive understanding for AI-powered multimodal BCIs.