I$^2$KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understanding

TL;DR

This paper introduces I$^2$KD-SLU, a novel framework for zero-shot cross-lingual spoken language understanding that leverages intra- and inter-knowledge distillation to improve intent detection and slot filling across languages.

Contribution

It proposes an intra-inter knowledge distillation framework that models mutual guidance between intent and slot predictions, advancing zero-shot cross-lingual SLU performance.

Findings

Significantly outperforms previous models on MultiATIS++ dataset.

Achieves new state-of-the-art accuracy in zero-shot cross-lingual SLU.

Demonstrates the effectiveness of intra-inter knowledge distillation in multilingual SLU.

Abstract

Spoken language understanding (SLU) typically includes two subtasks: intent detection and slot filling. Currently, it has achieved great success in high-resource languages, but it still remains challenging in low-resource languages due to the scarcity of labeled training data. Hence, there is a growing interest in zero-shot cross-lingual SLU. Despite of the success of existing zero-shot cross-lingual SLU models, most of them neglect to achieve the mutual guidance between intent and slots. To address this issue, we propose an Intra-Inter Knowledge Distillation framework for zero-shot cross-lingual Spoken Language Understanding (I$^2$KD-SLU) to model the mutual guidance. Specifically, we not only apply intra-knowledge distillation between intent predictions or slot predictions of the same utterance in different languages, but also apply inter-knowledge distillation between intent predictions and slot predictions of the same utterance. Our experimental results demonstrate that our proposed framework significantly improves the performance compared with the strong baselines and achieves the new state-of-the-art performance on the MultiATIS++ dataset, obtaining a significant improvement over the previous best model in overall accuracy.