Speech Synthesis along Perceptual Voice Quality Dimensions

TL;DR

This paper introduces a novel TTS system that uses a Conditional Continuous-Normalizing-Flow approach to control perceptual voice qualities like roughness and breathiness, learned from data rather than direct acoustic manipulation.

Contribution

It presents a new method for manipulating perceptual voice qualities in speech synthesis using a flow-based model trained on examples, avoiding direct acoustic feature manipulation.

Findings

System effectively modifies voice qualities as evaluated by phonetic experts.

Capable of handling both seen and unseen speakers.

Highlights strengths and areas for improvement in PVQ control.

Abstract

While expressive speech synthesis or voice conversion systems mainly focus on controlling or manipulating abstract prosodic characteristics of speech, such as emotion or accent, we here address the control of perceptual voice qualities (PVQs) recognized by phonetic experts, which are speech properties at a lower level of abstraction. The ability to manipulate PVQs can be a valuable tool for teaching speech pathologists in training or voice actors. In this paper, we integrate a Conditional Continuous-Normalizing-Flow-based method into a Text-to-Speech system to modify perceptual voice attributes on a continuous scale. Unlike previous approaches, our system avoids direct manipulation of acoustic correlates and instead learns from examples. We demonstrate the system's capability by manipulating four voice qualities: Roughness, breathiness, resonance and weight. Phonetic experts evaluated these modifications, both for seen and unseen speaker conditions. The results highlight both the system's strengths and areas for improvement.