Acoustically-Driven Phoneme Removal That Preserves Vocal Affect Cues

TL;DR

This paper introduces a novel acoustical method combining speech and EGG signals to remove linguistic content from speech, enabling the study of vocal affect without language interference, useful for clinical assessments.

Contribution

The proposed technique uniquely isolates paralinguistic affect cues by removing phonetic information while preserving vocal affect signals, using combined speech and EGG data.

Findings

High similarity in affect perception between original and transformed signals

Effective removal of linguistic content while retaining affect cues

Potential applications in clinical affect sensitivity testing

Abstract

In this paper, we propose a method for removing linguistic information from speech for the purpose of isolating paralinguistic indicators of affect. The immediate utility of this method lies in clinical tests of sensitivity to vocal affect that are not confounded by language, which is impaired in a variety of clinical populations. The method is based on simultaneous recordings of speech audio and electroglottographic (EGG) signals. The speech audio signal is used to estimate the average vocal tract filter response and amplitude envelop. The EGG signal supplies a direct correlate of voice source activity that is mostly independent of phonetic articulation. The dynamic energy of the speech audio and the average vocal tract filter are applied to the EGG signal create a third signal designed to capture as much paralinguistic information from the vocal production system as possible -- maximizing the retention of bioacoustic cues to affect -- while eliminating phonetic cues to verbal meaning. To evaluate the success of this method, we studied the perception of corresponding speech audio and transformed EGG signals in an affect rating experiment with online listeners. The results show a high degree of similarity in the perceived affect of matched signals, indicating that our method is effective.