Discovering Customer-Service Dialog System with Semi-Supervised Learning and Coarse-to-Fine Intent Detection

TL;DR

This paper presents a semi-supervised approach to develop task-oriented dialog systems by leveraging unlabelled data and coarse-to-fine intent detection, improving success rates and response coherence.

Contribution

It introduces a weakly supervised dataset creation method and a modular dialog system with coarse-to-fine intent classification, addressing data scarcity issues.

Findings

Higher dialog success rate achieved

More coherent response generation

Effective use of unlabelled dialogue data

Abstract

Task-oriented dialog(TOD) aims to assist users in achieving specific goals through multi-turn conversation. Recently, good results have been obtained based on large pre-trained models. However, the labeled-data scarcity hinders the efficient development of TOD systems at scale. In this work, we constructed a weakly supervised dataset based on a teacher/student paradigm that leverages a large collection of unlabelled dialogues. Furthermore, we built a modular dialogue system and integrated coarse-to-fine grained classification for user intent detection. Experiments show that our method can reach the dialog goal with a higher success rate and generate more coherent responses.