STUDIES: Corpus of Japanese Empathetic Dialogue Speech Towards Friendly Voice Agent

TL;DR

This paper introduces STUDIES, a Japanese empathetic dialogue speech corpus designed to enhance voice agents' naturalness by incorporating empathetic speech prosody and emotional context, enabling more human-like interactions.

Contribution

The paper presents a novel speech corpus and methodology for modeling empathetic dialogue in voice agents, including analysis and initial TTS experiments demonstrating improved naturalness.

Findings

Using interlocutor's emotion labels improves speech naturalness.

Conversational context embedding enhances empathetic speech synthesis.

Corpus construction methodology supports development of empathetic dialogue systems.

Abstract

We present STUDIES, a new speech corpus for developing a voice agent that can speak in a friendly manner. Humans naturally control their speech prosody to empathize with each other. By incorporating this "empathetic dialogue" behavior into a spoken dialogue system, we can develop a voice agent that can respond to a user more naturally. We designed the STUDIES corpus to include a speaker who speaks with empathy for the interlocutor's emotion explicitly. We describe our methodology to construct an empathetic dialogue speech corpus and report the analysis results of the STUDIES corpus. We conducted a text-to-speech experiment to initially investigate how we can develop more natural voice agent that can tune its speaking style corresponding to the interlocutor's emotion. The results show that the use of interlocutor's emotion label and conversational context embedding can produce speech with the same degree of naturalness as that synthesized by using the agent's emotion label. Our project page of the STUDIES corpus is http://sython.org/Corpus/STUDIES.