Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation

TL;DR

This paper introduces a bidirectional joint model for multiple intent detection and slot filling that leverages supervised contrastive learning and self-distillation, achieving superior accuracy on benchmark datasets.

Contribution

The paper presents a novel bidirectional joint model and a training method combining supervised contrastive learning and self-distillation for improved spoken language understanding.

Findings

Outperforms state-of-the-art models on MixATIS and MixSNIPS datasets.

Demonstrates the effectiveness of bidirectional design in joint intent detection and slot filling.

Shows that the proposed training method enhances model accuracy.

Abstract

Multiple intent detection and slot filling are two fundamental and crucial tasks in spoken language understanding. Motivated by the fact that the two tasks are closely related, joint models that can detect intents and extract slots simultaneously are preferred to individual models that perform each task independently. The accuracy of a joint model depends heavily on the ability of the model to transfer information between the two tasks so that the result of one task can correct the result of the other. In addition, since a joint model has multiple outputs, how to train the model effectively is also challenging. In this paper, we present a method for multiple intent detection and slot filling by addressing these challenges. First, we propose a bidirectional joint model that explicitly employs intent information to recognize slots and slot features to detect intents. Second, we introduce a novel method for training the proposed joint model using supervised contrastive learning and self-distillation. Experimental results on two benchmark datasets MixATIS and MixSNIPS show that our method outperforms state-of-the-art models in both tasks. The results also demonstrate the contributions of both bidirectional design and the training method to the accuracy improvement. Our source code is available at https://github.com/anhtunguyen98/BiSLU