Unified Knowledge Prompt Pre-training for Customer Service Dialogues

TL;DR

This paper introduces UFA, a unified pre-training framework that leverages knowledge prompts to improve customer service dialogue understanding and generation by unifying multiple tasks into a text-to-text format.

Contribution

The paper proposes a novel knowledge prompt pre-training approach that unifies various dialogue tasks and incorporates expert knowledge, enhancing performance on customer service dialogue benchmarks.

Findings

Significant improvements on NLU benchmarks.

Effective unification of multiple dialogue tasks.

Knowledge-driven prompts enhance learning efficiency.

Abstract

Dialogue bots have been widely applied in customer service scenarios to provide timely and user-friendly experience. These bots must classify the appropriate domain of a dialogue, understand the intent of users, and generate proper responses. Existing dialogue pre-training models are designed only for several dialogue tasks and ignore weakly-supervised expert knowledge in customer service dialogues. In this paper, we propose a novel unified knowledge prompt pre-training framework, UFA (\textbf{U}nified Model \textbf{F}or \textbf{A}ll Tasks), for customer service dialogues. We formulate all the tasks of customer service dialogues as a unified text-to-text generation task and introduce a knowledge-driven prompt strategy to jointly learn from a mixture of distinct dialogue tasks. We pre-train UFA on a large-scale Chinese customer service corpus collected from practical scenarios and get significant improvements on both natural language understanding (NLU) and natural language generation (NLG) benchmarks.