Toward Imagined Speech based Smart Communication System: Potential Applications on Metaverse Conditions

TL;DR

This paper explores the integration of imagined speech brain-computer interfaces with the metaverse to enable intuitive, brain-based communication and control, demonstrated through a virtual assistant for smart home management.

Contribution

It introduces a brain-to-speech system for real-world communication in the metaverse, including a demonstration of imagined speech for smart home control.

Findings

Achieved 46.54% accuracy in thirteen-class imagined speech classification.

Achieved 75.56% accuracy in binary imagined speech classification.

Supports the feasibility of imagined speech BCI for metaverse applications.

Abstract

Metaverse provides an alternative platform for human interaction in the virtual world. Since virtual platform holds few restrictions in changing the surrounding environments or the appearance of the avatars, it can serve as a platform that reflects human thoughts or even dreams at least in the metaverse world. When it is merged together with the current brain-computer interface (BCI) technology, which enables system control via brain signals, a new paradigm of human interaction through mind may be established in the metaverse conditions. Recent BCI systems are aiming to provide user-friendly and intuitive means of communication using brain signals. Imagined speech has become an alternative neuro-paradigm for communicative BCI since it relies directly on a person's speech production process, rather than using speech-unrelated neural activity as the means of communication. In this paper, we propose a brain-to-speech (BTS) system for real-world smart communication using brain signals. Also, we show a demonstration of imagined speech based smart home control through communication with a virtual assistant, which can be one of the future applications of brain-metaverse system. We performed pseudo-online analysis using imagined speech electroencephalography data of nine subjects to investigate the potential use of virtual BTS system in the real-world. Average accuracy of 46.54 % (chance level = 7.7 %) and 75.56 % (chance level = 50 %) was acquired in the thirteen-class and binary pseudo-online analysis, respectively. Our results support the potential of imagined speech based smart communication to be applied in the metaverse world.