MIDAS: A Dialog Act Annotation Scheme for Open Domain Human Machine Spoken Conversations

TL;DR

This paper introduces MIDAS, a hierarchical dialog act annotation scheme tailored for open-domain human-machine spoken conversations, enabling better understanding with limited machine comprehension capabilities.

Contribution

The paper presents MIDAS, a novel annotation scheme designed specifically for open-domain human-machine dialogues, and demonstrates its effectiveness through a large annotated dataset and transfer learning models.

Findings

Achieved an F1 score of 0.79 in dialog act prediction.

Created a large dataset of 24,000 annotated utterances.

Demonstrated the scheme's applicability in real-world scenarios.

Abstract

Dialog act prediction is an essential language comprehension task for both dialog system building and discourse analysis. Previous dialog act schemes, such as SWBD-DAMSL, are designed for human-human conversations, in which conversation partners have perfect language understanding ability. In this paper, we design a dialog act annotation scheme, MIDAS (Machine Interaction Dialog Act Scheme), targeted on open-domain human-machine conversations. MIDAS is designed to assist machines which have limited ability to understand their human partners. MIDAS has a hierarchical structure and supports multi-label annotations. We collected and annotated a large open-domain human-machine spoken conversation dataset (consists of 24K utterances). To show the applicability of the scheme, we leverage transfer learning methods to train a multi-label dialog act prediction model and reach an F1 score of 0.79.