A Chit-Chats Enhanced Task-Oriented Dialogue Corpora for Fuse-Motive Conversation Systems

TL;DR

This paper introduces a new Chinese multi-turn dialogue dataset combining task-oriented and chit-chat motives, addressing the limitations of existing datasets and proposing formalization and evaluation methods for fuse-motive conversations.

Contribution

It presents CCET, a Chinese fuse-motive dialogue dataset, and proposes formalization and evaluation metrics for multi-turn dialogues blending TOD and CC.

Findings

CCET dataset effectively captures diverse user motives.

Proposed formalization improves dialogue system training.

Evaluation metrics enhance assessment of fuse-motive dialogues.

Abstract

The goal of building intelligent dialogue systems has largely been separately pursued under two motives: task-oriented dialogue (TOD) systems, and open-domain systems for chit-chat (CC). Although previous TOD dialogue systems work well in the testing sets of benchmarks, they would lead to undesirable failure when being exposed to natural scenarios in practice, where user utterances can be of high motive-diversity that fusing both TOD and CC in multi-turn interaction. Since an industrial TOD system should be able to converse with the user between TOD and CC motives, constructing a fuse-motive dialogue dataset that contains both TOD or CC is important. Most prior work relies on crowd workers to collect and annotate large scale dataset and is restricted to English language setting. Our work, on the contrary, addresses this problem in a more effective way and releases a multi-turn dialogues dataset called CCET (Chinese Chat-Enhanced-Task). Meanwhile, we also propose a line of fuse-motive dialogues formalization approach, along with several evaluation metrics for TOD sessions that are integrated by CC utterances.