Energy-based Unknown Intent Detection with Data Manipulation

TL;DR

This paper introduces an energy-based method for detecting unknown intents in dialogue systems, utilizing a novel data manipulation framework to generate high-quality OOD examples, leading to improved detection performance.

Contribution

The paper proposes a new energy-based approach combined with a data manipulation framework called GOT to generate high-quality OOD data for better unknown intent detection.

Findings

Achieves state-of-the-art results on benchmark datasets.

Effective energy score alignment with input density.

GOT improves OOD detection accuracy significantly.

Abstract

Unknown intent detection aims to identify the out-of-distribution (OOD) utterance whose intent has never appeared in the training set. In this paper, we propose using energy scores for this task as the energy score is theoretically aligned with the density of the input and can be derived from any classifier. However, high-quality OOD utterances are required during the training stage in order to shape the energy gap between OOD and in-distribution (IND), and these utterances are difficult to collect in practice. To tackle this problem, we propose a data manipulation framework to Generate high-quality OOD utterances with importance weighTs (GOT). Experimental results show that the energy-based detector fine-tuned by GOT can achieve state-of-the-art results on two benchmark datasets.