Contextual Dialogue Act Classification for Open-Domain Conversational Agents

TL;DR

This paper introduces CDAC, a deep learning model that uses transfer learning to improve dialogue act classification in open-domain conversational agents by leveraging human-human dialogue data and fine-tuning on human-machine conversations.

Contribution

The paper presents a novel transfer learning approach, CDAC, for contextual dialogue act classification that adapts models trained on human-human data to human-machine dialogues, enhancing accuracy.

Findings

CDAC outperforms previous utterance-level baselines by 8% on Switchboard.

Fine-tuning on small human-machine datasets improves prediction accuracy.

CDAC achieves comparable results to state-of-the-art contextual DA classifiers.

Abstract

Classifying the general intent of the user utterance in a conversation, also known as Dialogue Act (DA), e.g., open-ended question, statement of opinion, or request for an opinion, is a key step in Natural Language Understanding (NLU) for conversational agents. While DA classification has been extensively studied in human-human conversations, it has not been sufficiently explored for the emerging open-domain automated conversational agents. Moreover, despite significant advances in utterance-level DA classification, full understanding of dialogue utterances requires conversational context. Another challenge is the lack of available labeled data for open-domain human-machine conversations. To address these problems, we propose a novel method, CDAC (Contextual Dialogue Act Classifier), a simple yet effective deep learning approach for contextual dialogue act classification. Specifically, we use transfer learning to adapt models trained on human-human conversations to predict dialogue acts in human-machine dialogues. To investigate the effectiveness of our method, we train our model on the well-known Switchboard human-human dialogue dataset, and fine-tune it for predicting dialogue acts in human-machine conversation data, collected as part of the Amazon Alexa Prize 2018 competition. The results show that the CDAC model outperforms an utterance-level state of the art baseline by 8.0% on the Switchboard dataset, and is comparable to the latest reported state-of-the-art contextual DA classification results. Furthermore, our results show that fine-tuning the CDAC model on a small sample of manually labeled human-machine conversations allows CDAC to more accurately predict dialogue acts in real users' conversations, suggesting a promising direction for future improvements.