Evaluating and Rewarding LALMs for Expressive Role-Play TTS via Mean Continuation Log-Probability

TL;DR

This paper introduces Mean Continuation Log-Probability (MCLP), a novel metric and reward for evaluating and improving expressive role-play TTS with Large Audio Language Models, ensuring stylistic consistency and better alignment with instructions.

Contribution

We propose MCLP as an objective metric and reinforcement learning reward to enhance stylistic consistency in LALM-based Role-Play TTS, validated on a new annotated dataset.

Findings

MCLP effectively quantifies stylistic consistency.

Reinforcement learning with MCLP improves style alignment.

Our method outperforms baseline models on objective and subjective metrics.

Abstract

Recent advances in Large Audio Language Models (LALMs) have extended Text-to-Speech (TTS) to interactive role-play scenarios, which demand high expressiveness and strict adherence to role-play instructions. However, existing models struggle to maintain stylistic consistency with character profiles and scene descriptions across multi-turn dialogues. A critical bottleneck is the lack of objective metrics for quantifying speaking style. To bridge this gap, we propose Mean Continuation Log-Probability (MCLP) as both an evaluation metric and a reward signal, validated on LALM-based Role-Play TTS (RP-TTS) tasks. Critically, we leverage the In-Context Learning capability of pre-trained LALMs to formulate MCLP via a continuation log-probability prediction. This metric quantifies stylistic consistency by measuring the likelihood of the ground-truth speech conditioned on the generated speech. Furthermore, we employ MCLP as a reinforcement learning reward to enhance the style alignment between generated speech and Role-Play instructions. To facilitate evaluation, we construct an RP-TTS dataset with rich scene and character annotations. Experimental results demonstrate that our method significantly outperforms strong LALM baselines on both objective and subjective metrics.