Toward Objective and Interpretable Prosody Evaluation in Text-to-Speech: A Linguistically Motivated Approach

TL;DR

This paper presents a linguistically motivated, objective framework for evaluating TTS prosody that correlates well with perceptual ratings and offers interpretability for diagnosing system weaknesses.

Contribution

It introduces a semi-automatic, linguistically informed evaluation method that captures both prosodic event placement and cue realization, improving upon traditional metrics.

Findings

Strong correlation with perceptual MOS ratings

Reveals model-specific prosodic weaknesses

Provides interpretable metrics for prosody evaluation

Abstract

Prosody is essential for speech technology, shaping comprehension, naturalness, and expressiveness. However, current text-to-speech (TTS) systems still struggle to accurately capture human-like prosodic variation, in part because existing evaluation methods for prosody remain limited. Traditional metrics like Mean Opinion Score (MOS) are resource-intensive, inconsistent, and offer little insight into why a system sounds unnatural. This study introduces a linguistically informed, semi-automatic framework for evaluating TTS prosody through a two-tier architecture that mirrors human prosodic organization. The method uses quantitative linguistic criteria to evaluate synthesized speech against human speech corpora across multiple acoustic dimensions. By integrating discrete and continuous prosodic measures, it provides objective and interpretable metrics of both event placement and cue realization, while accounting for the natural variability observed across speakers and prosodic cues. Results show strong correlations with perceptual MOS ratings while revealing model-specific weaknesses that traditional perceptual tests alone cannot capture. This approach provides a principled path toward diagnosing, benchmarking, and ultimately improving the prosodic naturalness of next-generation TTS systems.