Speechworthy Instruction-tuned Language Models

TL;DR

This paper enhances instruction-tuned language models for speech by developing new prompting strategies and a speech-based preference learning approach, significantly improving speech-suitability in generated responses through combined methods.

Contribution

It introduces a novel speech-based preference dataset and demonstrates that combining prompting and preference learning optimally improves speech-suitability of language models.

Findings

Combined prompting and preference learning achieve 76.2% win rate in preferences.

Both methods independently improve speech-suitability.

Analyses reveal distinct contributions of each method to response quality.

Abstract

Current instruction-tuned language models are exclusively trained with textual preference data and thus are often not aligned with the unique requirements of other modalities, such as speech. To better align language models with the speech domain, we explore (i) prompting strategies grounded in radio-industry best practices and (ii) preference learning using a novel speech-based preference data of 20K samples, generated with a wide spectrum of prompts that induce varying dimensions of speech-suitability and labeled by annotators who listen to response pairs. Both human and automatic evaluation show that both prompting and preference learning increase the speech-suitability of popular instruction-tuned LLMs. Interestingly, we find that prompting and preference learning can be additive; combining them achieves the best win rates in head-to-head comparison, resulting in responses that are preferred or tied to the base model in 76.2% of comparisons on average. Lastly, we share lexical, syntactical, and qualitative analyses to showcase how each method contributes to improving the speech-suitability of generated responses.