Decoding Linguistic Representations of Human Brain

TL;DR

This paper presents a comprehensive taxonomy of brain-to-language decoding methods, integrating neuroscience and deep learning to advance understanding of linguistic representations in the brain and aid brain-computer interfaces.

Contribution

It introduces a unified taxonomy for decoding linguistic information from brain activity, bridging neuroscience and deep learning research.

Findings

Taxonomy of brain-to-language decoding methods

Integration of neuroscience and deep learning approaches

Potential applications for brain-computer interfaces

Abstract

Language, as an information medium created by advanced organisms, has always been a concern of neuroscience regarding how it is represented in the brain. Decoding linguistic representations in the evoked brain has shown groundbreaking achievements, thanks to the rapid improvement of neuroimaging, medical technology, life sciences and artificial intelligence. In this work, we present a taxonomy of brain-to-language decoding of both textual and speech formats. This work integrates two types of research: neuroscience focusing on language understanding and deep learning-based brain decoding. Generating discernible language information from brain activity could not only help those with limited articulation, especially amyotrophic lateral sclerosis (ALS) patients but also open up a new way for the next generation's brain-computer interface (BCI). This article will help brain scientists and deep-learning researchers to gain a bird's eye view of fine-grained language perception, and thus facilitate their further investigation and research of neural process and language decoding.