Novel Slot Detection: A Benchmark for Discovering Unknown Slot Types in the Task-Oriented Dialogue System

TL;DR

This paper introduces a new task called Novel Slot Detection (NSD) for task-oriented dialogue systems, aiming to identify unknown slot types to improve system robustness, along with datasets, baselines, and a benchmark for future research.

Contribution

It proposes the NSD task, creates two datasets, establishes a benchmark, and provides baseline models and analysis for discovering unknown slot types.

Findings

Constructed two public NSD datasets.

Established strong baseline models for NSD.

Conducted comprehensive experiments and analysis.

Abstract

Existing slot filling models can only recognize pre-defined in-domain slot types from a limited slot set. In the practical application, a reliable dialogue system should know what it does not know. In this paper, we introduce a new task, Novel Slot Detection (NSD), in the task-oriented dialogue system. NSD aims to discover unknown or out-of-domain slot types to strengthen the capability of a dialogue system based on in-domain training data. Besides, we construct two public NSD datasets, propose several strong NSD baselines, and establish a benchmark for future work. Finally, we conduct exhaustive experiments and qualitative analysis to comprehend key challenges and provide new guidance for future directions.