New Intent Discovery with Pre-training and Contrastive Learning

TL;DR

This paper introduces a novel approach combining multi-task pre-training and contrastive learning to improve new intent discovery, effectively leveraging unlabeled data for better representation and clustering in dialogue systems.

Contribution

It proposes a multi-task pre-training strategy and a new contrastive loss to enhance intent discovery from unlabeled data, outperforming existing methods.

Findings

Outperforms state-of-the-art methods by a large margin.

Effective in both unsupervised and semi-supervised scenarios.

Demonstrates high effectiveness on three intent recognition benchmarks.

Abstract

New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes. It is a critical task for the development and service expansion of a practical dialogue system. Despite its importance, this problem remains under-explored in the literature. Existing approaches typically rely on a large amount of labeled utterances and employ pseudo-labeling methods for representation learning and clustering, which are label-intensive, inefficient, and inaccurate. In this paper, we provide new solutions to two important research questions for new intent discovery: (1) how to learn semantic utterance representations and (2) how to better cluster utterances. Particularly, we first propose a multi-task pre-training strategy to leverage rich unlabeled data along with external labeled data for representation learning. Then, we design a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering. Extensive experiments on three intent recognition benchmarks demonstrate the high effectiveness of our proposed method, which outperforms state-of-the-art methods by a large margin in both unsupervised and semi-supervised scenarios. The source code will be available at https://github.com/zhang-yu-wei/MTP-CLNN.