A Novel Task-Oriented Text Corpus in Silent Speech Recognition and its Natural Language Generation Construction Method

TL;DR

This paper introduces a new task-oriented text corpus for silent speech recognition using EEG, along with a hybrid natural language generation method that improves data diversity and linguistic quality, aiding SSR development.

Contribution

It presents a novel task-oriented text corpus for SSR and a hybrid NLG construction method combining templates and neural networks, enhancing data quality and diversity.

Findings

Hybrid construction outperforms pure methods in SSR tasks.

Generated corpus improves speech recognition performance.

Method ensures high linguistic quality and diversity.

Abstract

Millions of people with severe speech disorders around the world may regain their communication capabilities through techniques of silent speech recognition (SSR). Using electroencephalography (EEG) as a biomarker for speech decoding has been popular for SSR. However, the lack of SSR text corpus has impeded the development of this technique. Here, we construct a novel task-oriented text corpus, which is utilized in the field of SSR. In the process of construction, we propose a task-oriented hybrid construction method based on natural language generation algorithm. The algorithm focuses on the strategy of data-to-text generation, and has two advantages including linguistic quality and high diversity. These two advantages use template-based method and deep neural networks respectively. In an SSR experiment with the generated text corpus, analysis results show that the performance of our hybrid construction method outperforms the pure method such as template-based natural language generation or neural natural language generation models.