Disentangling segmental and prosodic factors to non-native speech comprehensibility

TL;DR

This paper introduces a novel accent conversion system that disentangles segmental and prosodic features, enabling independent manipulation to study their effects on non-native speech comprehensibility and social attitudes.

Contribution

The system uniquely separates segmental and prosodic features in non-native speech, allowing targeted manipulation and analysis of their individual impacts on comprehensibility.

Findings

Segmental features have a larger impact on comprehensibility than prosody.

Vector quantization improves prosody transfer and voice similarity.

Perceptual tests quantify contributions of speech features to intelligibility.

Abstract

Current accent conversion (AC) systems do not disentangle the two main sources of non-native accent: segmental and prosodic characteristics. Being able to manipulate a non-native speaker's segmental and/or prosodic channels independently is critical to quantify how these two channels contribute to speech comprehensibility and social attitudes. We present an AC system that not only decouples voice quality from accent, but also disentangles the latter into its segmental and prosodic characteristics. The system is able to generate accent conversions that combine (1) the segmental characteristics from a source utterance, (2) the voice characteristics from a target utterance, and (3) the prosody of a reference utterance. We show that vector quantization of acoustic embeddings and removal of consecutive duplicated codewords allows the system to transfer prosody and improve voice similarity. We conduct perceptual listening tests to quantify the individual contributions of segmental features and prosody on the perceived comprehensibility of non-native speech. Our results indicate that, contrary to prior research in non-native speech, segmental features have a larger impact on comprehensibility than prosody. The proposed AC system may also be used to study how segmental and prosody cues affect social attitudes towards non-native speech.