STIL -- Simultaneous Slot Filling, Translation, Intent Classification, and Language Identification: Initial Results using mBART on MultiATIS++

TL;DR

This paper introduces STIL, a new multilingual NLU task combining slot filling, translation, intent classification, and language identification, demonstrating the effectiveness of fine-tuned mBART on MultiATIS++ with minimal performance loss.

Contribution

It proposes the STIL task and shows that fine-tuned mBART can perform multiple NLU tasks simultaneously with competitive accuracy and minimal degradation when translating.

Findings

mBART achieves comparable intent classification accuracy to state-of-the-art systems.

Simultaneous translation causes only slight performance degradation.

The approach reduces downstream system complexity by enabling monolingual components.

Abstract

Slot-filling, Translation, Intent classification, and Language identification, or STIL, is a newly-proposed task for multilingual Natural Language Understanding (NLU). By performing simultaneous slot filling and translation into a single output language (English in this case), some portion of downstream system components can be monolingual, reducing development and maintenance cost. Results are given using the multilingual BART model (Liu et al., 2020) fine-tuned on 7 languages using the MultiATIS++ dataset. When no translation is performed, mBART's performance is comparable to the current state of the art system (Cross-Lingual BERT by Xu et al. (2020)) for the languages tested, with better average intent classification accuracy (96.07% versus 95.50%) but worse average slot F1 (89.87% versus 90.81%). When simultaneous translation is performed, average intent classification accuracy degrades by only 1.7% relative and average slot F1 degrades by only 1.2% relative.