An Introduction to Silent Paralinguistics

TL;DR

This paper introduces silent paralinguistics, a new research area focused on extracting emotional and affective cues from silent speech biosignals to enhance silent speech interfaces and natural communication.

Contribution

It presents a comprehensive survey of methods and research strategies for deriving paralinguistic information from silent speech, addressing a significant gap in the field.

Findings

Silent paralinguistics aims to predict affective states from silent speech biosignals.

The survey highlights existing methods and identifies research gaps in silent speech paralinguistics.

Integration of paralinguistic cues into silent speech interfaces can improve naturalness of communication.

Abstract

The ability to speak is an inherent part of human nature and fundamental to our existence as a social species. Unfortunately, this ability can be restricted in certain situations, such as for individuals who have lost their voice or in environments where speaking aloud is unsuitable. Additionally, some people may prefer not to speak audibly due to privacy concerns. For such cases, silent speech interfaces have been proposed, which focus on processing biosignals corresponding to silently produced speech. These interfaces enable synthesis of audible speech from biosignals that are produced when speaking silently and recognition aka decoding of biosignals into text that corresponds to the silently produced speech. While recognition and synthesis of silent speech has been a prominent focus in many research studies, there is a significant gap in deriving paralinguistic information such as affective states from silent speech. To fill this gap, we propose Silent Paralinguistics, aiming to predict paralinguistic information from silent speech and ultimately integrate it into the reconstructed audible voice for natural communication. This survey provides a comprehensive look at methods, research strategies, and objectives within the emerging field of silent paralinguistics.