Quantifying the perceptual value of lexical and non-lexical channels in speech

TL;DR

This paper introduces a new method to quantify the impact of non-lexical speech cues on listener expectations, demonstrating their significant role in dialogue perception beyond lexical content.

Contribution

It presents a generalized paradigm for measuring non-lexical information's value in dialogue without relying on specific lexical variations or acoustic manipulations.

Findings

Non-lexical cues influence listener expectations consistently.

Non-lexical information increases consensus among listeners.

Even when less discriminative, non-lexical cues affect perception significantly.

Abstract

Speech is a fundamental means of communication that can be seen to provide two channels for transmitting information: the lexical channel of which words are said, and the non-lexical channel of how they are spoken. Both channels shape listener expectations of upcoming communication; however, directly quantifying their relative effect on expectations is challenging. Previous attempts require spoken variations of lexically-equivalent dialogue turns or conspicuous acoustic manipulations. This paper introduces a generalised paradigm to study the value of non-lexical information in dialogue across unconstrained lexical content. By quantifying the perceptual value of the non-lexical channel with both accuracy and entropy reduction, we show that non-lexical information produces a consistent effect on expectations of upcoming dialogue: even when it leads to poorer discriminative turn judgements than lexical content alone, it yields higher consensus among participants.