Multilingual and cross-lingual document classification: A meta-learning approach

TL;DR

This paper introduces a meta-learning approach for multilingual and cross-lingual document classification, effectively handling low-resource languages and limited data scenarios, and achieving state-of-the-art results.

Contribution

It presents a novel meta-learning method tailored for under-resourced languages, with a simple adjustment that improves stability and performance in multilingual classification tasks.

Findings

Meta-learning excels with heterogeneous task distributions.

Proposed adjustment enhances learning stability.

Achieves new state-of-the-art on several languages.

Abstract

The great majority of languages in the world are considered under-resourced for the successful application of deep learning methods. In this work, we propose a meta-learning approach to document classification in limited-resource setting and demonstrate its effectiveness in two different settings: few-shot, cross-lingual adaptation to previously unseen languages; and multilingual joint training when limited target-language data is available during training. We conduct a systematic comparison of several meta-learning methods, investigate multiple settings in terms of data availability and show that meta-learning thrives in settings with a heterogeneous task distribution. We propose a simple, yet effective adjustment to existing meta-learning methods which allows for better and more stable learning, and set a new state of the art on several languages while performing on-par on others, using only a small amount of labeled data.