RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training

TL;DR

RSVP is a self-supervised pre-training framework for customer intent detection that leverages agent responses, improving accuracy over existing models by incorporating response relations during training.

Contribution

The paper introduces RSVP, a novel two-stage pre-training method utilizing agent responses for improved customer intent detection in task-oriented dialogues.

Findings

RSVP outperforms state-of-the-art baselines by up to 4.95% in accuracy.

Incorporating agent responses enhances intent detection performance.

The approach is validated on two real-world customer service datasets.

Abstract

The dialogue systems in customer services have been developed with neural models to provide users with precise answers and round-the-clock support in task-oriented conversations by detecting customer intents based on their utterances. Existing intent detection approaches have highly relied on adaptively pre-training language models with large-scale datasets, yet the predominant cost of data collection may hinder their superiority. In addition, they neglect the information within the conversational responses of the agents, which have a lower collection cost, but are significant to customer intent as agents must tailor their replies based on the customers' intent. In this paper, we propose RSVP, a self-supervised framework dedicated to task-oriented dialogues, which utilizes agent responses for pre-training in a two-stage manner. Specifically, we introduce two pre-training tasks to incorporate the relations of utterance-response pairs: 1) Response Retrieval by selecting a correct response from a batch of candidates, and 2) Response Generation by mimicking agents to generate the response to a given utterance. Our benchmark results for two real-world customer service datasets show that RSVP significantly outperforms the state-of-the-art baselines by 4.95% for accuracy, 3.4% for MRR@3, and 2.75% for MRR@5 on average. Extensive case studies are investigated to show the validity of incorporating agent responses into the pre-training stage.