CrossAligner & Co: Zero-Shot Transfer Methods for Task-Oriented Cross-lingual Natural Language Understanding

TL;DR

This paper introduces CrossAligner, a zero-shot transfer method for task-oriented cross-lingual natural language understanding that leverages unlabelled parallel data to improve multilingual performance.

Contribution

We propose CrossAligner, a novel alignment-based approach for zero-shot transfer in multilingual tasks, outperforming existing methods on multiple datasets and languages.

Findings

Several methods exceed state-of-the-art scores.

Fine-tuned models transfer task knowledge effectively.

Qualitative analysis confirms strong zero-shot transfer capabilities.

Abstract

Task-oriented personal assistants enable people to interact with a host of devices and services using natural language. One of the challenges of making neural dialogue systems available to more users is the lack of training data for all but a few languages. Zero-shot methods try to solve this issue by acquiring task knowledge in a high-resource language such as English with the aim of transferring it to the low-resource language(s). To this end, we introduce CrossAligner, the principal method of a variety of effective approaches for zero-shot cross-lingual transfer based on learning alignment from unlabelled parallel data. We present a quantitative analysis of individual methods as well as their weighted combinations, several of which exceed state-of-the-art (SOTA) scores as evaluated across nine languages, fifteen test sets and three benchmark multilingual datasets. A detailed qualitative error analysis of the best methods shows that our fine-tuned language models can zero-shot transfer the task knowledge better than anticipated.