MELD-ST: An Emotion-aware Speech Translation Dataset

TL;DR

This paper introduces the MELD-ST dataset for emotion-aware speech translation, emphasizing the importance of emotion in translation quality and providing a new resource for developing more emotionally intelligent translation systems.

Contribution

The paper presents the MELD-ST dataset, a new emotion-annotated speech translation dataset for English-Japanese and English-German, enabling research on emotion-aware translation.

Findings

Fine-tuning with emotion labels can improve translation performance.

Baseline experiments demonstrate the dataset's potential for emotion-aware translation.

Highlighting the need for further research in emotion-aware speech translation.

Abstract

Emotion plays a crucial role in human conversation. This paper underscores the significance of considering emotion in speech translation. We present the MELD-ST dataset for the emotion-aware speech translation task, comprising English-to-Japanese and English-to-German language pairs. Each language pair includes about 10,000 utterances annotated with emotion labels from the MELD dataset. Baseline experiments using the SeamlessM4T model on the dataset indicate that fine-tuning with emotion labels can enhance translation performance in some settings, highlighting the need for further research in emotion-aware speech translation systems.