Voice Impression Control in Zero-Shot TTS

TL;DR

This paper introduces a zero-shot TTS method that controls voice impressions using a low-dimensional vector, enabling natural language-based impression specification and demonstrating effectiveness through evaluations.

Contribution

The paper presents a novel zero-shot TTS approach that uses a low-dimensional vector for impression control, including a method to generate this vector from natural language descriptions.

Findings

Effective impression control demonstrated in evaluations

Natural language-based impression generation enabled

No manual optimization needed for impression specification

Abstract

Para-/non-linguistic information in speech is pivotal in shaping the listeners' impression. Although zero-shot text-to-speech (TTS) has achieved high speaker fidelity, modulating subtle para-/non-linguistic information to control perceived voice characteristics, i.e., impressions, remains challenging. We have therefore developed a voice impression control method in zero-shot TTS that utilizes a low-dimensional vector to represent the intensities of various voice impression pairs (e.g., dark-bright). The results of both objective and subjective evaluations have demonstrated our method's effectiveness in impression control. Furthermore, generating this vector via a large language model enables target-impression generation from a natural language description of the desired impression, thus eliminating the need for manual optimization. Audio examples are available on our demo page (https://ntt-hilab-gensp.github.io/is2025voiceimpression/).