Covertly improving intelligibility with data-driven adaptations of speech timing

TL;DR

This study demonstrates that targeted, data-driven adjustments to speech timing can significantly improve speech intelligibility for diverse listeners, often without their awareness, and introduces a novel speech synthesis method to implement these findings.

Contribution

The paper introduces a data-driven approach to modify speech timing in synthesis, enhancing comprehension for both native and non-native listeners under challenging conditions.

Findings

Targeted speech-rate adjustments improve comprehension in noisy environments.

Listeners are unaware of the benefits of targeted slowing, perceiving it as clearer.

A new text-to-speech algorithm replicates beneficial temporal structures in speech.

Abstract

Human talkers often address listeners with language-comprehension challenges, such as hard-of-hearing or non-native adults, by globally slowing down their speech. However, it remains unclear whether this strategy actually makes speech more intelligible. Here, we take advantage of recent advancements in machine-generated speech allowing more precise control of speech rate in order to systematically examine how targeted speech-rate adjustments may improve comprehension. We first use reverse-correlation experiments to show that the temporal influence of speech rate prior to a target vowel contrast (ex. the tense-lax distinction) in fact manifests in a scissor-like pattern, with opposite effects in early versus late context windows; this pattern is remarkably stable both within individuals and across native L1-English listeners and L2-English listeners with French, Mandarin, and Japanese L1s. Second, we show that this speech rate structure not only facilitates L2 listeners' comprehension of the target vowel contrast, but that native listeners also rely on this pattern in challenging acoustic conditions. Finally, we build a data-driven text-to-speech algorithm that replicates this temporal structure on novel speech sequences. Across a variety of sentences and vowel contrasts, listeners remained unaware that such targeted slowing improved word comprehension. Strikingly, participants instead judged the common strategy of global slowing as clearer, even though it actually increased comprehension errors. Together, these results show that targeted adjustments to speech rate significantly aid intelligibility under challenging conditions, while often going unnoticed. More generally, this paper provides a data-driven methodology to improve the accessibility of machine-generated speech which can be extended to other aspects of speech comprehension and a wide variety of listeners and environments.