Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model

TL;DR

This paper introduces Bert-Joint, a multilingual, recurrence-less model for intent detection and slot filling that performs well across languages and data availability levels, simplifying multilingual spoken language understanding.

Contribution

The paper presents Bert-Joint, a novel multilingual joint model for intent detection and slot filling that does not rely on recurrence, demonstrating strong performance across languages and datasets.

Findings

High performance on English benchmarks with limited data

Effective cross-lingual transfer to Italian without model modifications

Simplifies multilingual spoken language understanding models

Abstract

Intent Detection and Slot Filling are two pillar tasks in Spoken Natural Language Understanding. Common approaches adopt joint Deep Learning architectures in attention-based recurrent frameworks. In this work, we aim at exploiting the success of "recurrence-less" models for these tasks. We introduce Bert-Joint, i.e., a multi-lingual joint text classification and sequence labeling framework. The experimental evaluation over two well-known English benchmarks demonstrates the strong performances that can be obtained with this model, even when few annotated data is available. Moreover, we annotated a new dataset for the Italian language, and we observed similar performances without the need for changing the model.