Cross-lingual Adaption Model-Agnostic Meta-Learning for Natural Language Understanding

TL;DR

This paper introduces XLA-MAML, a meta-learning approach that enhances cross-lingual transfer in NLP tasks by directly adapting across languages, showing improved zero-shot and few-shot performance.

Contribution

It proposes a novel cross-lingual meta-learning method, XLA-MAML, that addresses limitations of previous models by enabling direct adaptation across languages during meta-learning.

Findings

Effective in zero-shot and few-shot settings

Improves performance across multiple languages and tasks

Analyzes cross-lingual sampling strategies

Abstract

Meta learning with auxiliary languages has demonstrated promising improvements for cross-lingual natural language processing. However, previous studies sample the meta-training and meta-testing data from the same language, which limits the ability of the model for cross-lingual transfer. In this paper, we propose XLA-MAML, which performs direct cross-lingual adaption in the meta-learning stage. We conduct zero-shot and few-shot experiments on Natural Language Inference and Question Answering. The experimental results demonstrate the effectiveness of our method across different languages, tasks, and pretrained models. We also give analysis on various cross-lingual specific settings for meta-learning including sampling strategy and parallelism.