A Generative Model for Joint Multiple Intent Detection and Slot Filling

TL;DR

This paper introduces a generative model with an attention-over-attention decoder for simultaneous multiple intent detection and slot filling in dialogue systems, addressing multi-intent challenges and outperforming existing methods.

Contribution

It proposes a novel generative framework with an attention-over-attention decoder for multi-intent SLU and creates new datasets using BERT's NSP head for evaluation.

Findings

Achieves state-of-the-art results on MixATIS and MixSNIPS datasets.

Effectively handles multiple intents and slot filling in real-world dialogues.

Constructed new multi-intent SLU datasets from single-intent data.

Abstract

In task-oriented dialogue systems, spoken language understanding (SLU) is a critical component, which consists of two sub-tasks, intent detection and slot filling. Most existing methods focus on the single-intent SLU, where each utterance only has one intent. However, in real-world scenarios users usually express multiple intents in an utterance, which poses a challenge for existing dialogue systems and datasets. In this paper, we propose a generative framework to simultaneously address multiple intent detection and slot filling. In particular, an attention-over-attention decoder is proposed to handle the variable number of intents and the interference between the two sub-tasks by incorporating an inductive bias into the process of multi-task learning. Besides, we construct two new multi-intent SLU datasets based on single-intent utterances by taking advantage of the next sentence prediction (NSP) head of the BERT model. Experimental results demonstrate that our proposed attention-over-attention generative model achieves state-of-the-art performance on two public datasets, MixATIS and MixSNIPS, and our constructed datasets.