Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language

TL;DR

This paper introduces Emotion Transcription in Conversation (ETC), a new task and dataset for capturing complex emotional states in dialogue through natural language descriptions, advancing emotion understanding in conversational AI.

Contribution

The paper presents a novel task and a Japanese dataset with natural language emotion annotations, enabling more nuanced emotion recognition in dialogue systems.

Findings

Fine-tuning improves model performance on ETC

Current models struggle with implicit emotional states

The dataset facilitates more expressive emotion understanding

Abstract

Emotion Recognition in Conversation (ERC) is critical for enabling natural human-machine interactions. However, existing methods predominantly employ categorical or dimensional emotion annotations, which often fail to adequately represent complex, subtle, or culturally specific emotional nuances. To overcome this limitation, we propose a novel task named Emotion Transcription in Conversation (ETC). This task focuses on generating natural language descriptions that accurately reflect speakers' emotional states within conversational contexts. To address the ETC, we constructed a Japanese dataset comprising text-based dialogues annotated with participants' self-reported emotional states, described in natural language. The dataset also includes emotion category labels for each transcription, enabling quantitative analysis and its application to ERC. We benchmarked baseline models, finding that while fine-tuning on our dataset enhances model performance, current models still struggle to infer implicit emotional states. The ETC task will encourage further research into more expressive emotion understanding in dialogue. The dataset is publicly available at https://github.com/UEC-InabaLab/ETCDataset.