The Massively Multilingual Natural Language Understanding 2022 (MMNLU-22) Workshop and Competition

TL;DR

This paper introduces a new multilingual NLU dataset, organizes a challenge and workshop to advance multilingual NLU technology, and reports on progress and findings from these initiatives.

Contribution

It provides a large multilingual dataset, a competitive challenge, and a workshop to foster research and improve transferability of NLU models across languages.

Findings

Progress in multilingual NLU transferability

Enhanced dataset for voice assistant evaluation

Community engagement through challenge and workshop

Abstract

Despite recent progress in Natural Language Understanding (NLU), the creation of multilingual NLU systems remains a challenge. It is common to have NLU systems limited to a subset of languages due to lack of available data. They also often vary widely in performance. We launch a three-phase approach to address the limitations in NLU and help propel NLU technology to new heights. We release a 52 language dataset called the Multilingual Amazon SLU resource package (SLURP) for Slot-filling, Intent classification, and Virtual assistant Evaluation, or MASSIVE, in an effort to address parallel data availability for voice assistants. We organize the Massively Multilingual NLU 2022 Challenge to provide a competitive environment and push the state-of-the art in the transferability of models into other languages. Finally, we host the first Massively Multilingual NLU workshop which brings these components together. The MMNLU workshop seeks to advance the science behind multilingual NLU by providing a platform for the presentation of new research in the field and connecting teams working on this research direction. This paper summarizes the dataset, workshop and the competition and the findings of each phase.