SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection

TL;DR

The SIGMORPHON 2020 shared task evaluated the ability of systems to perform morphological inflection across 90 typologically diverse languages, highlighting neural models' effectiveness and the benefits of data augmentation, with some non-neural methods excelling in low-resource scenarios.

Contribution

This study introduces a large-scale multilingual morphological inflection challenge, comparing neural and non-neural approaches across many languages and resource levels, revealing insights into model generalization and data strategies.

Findings

Neural models dominated top performance with transformers and multilingual RNNs.

Data augmentation and ensemble methods improved low-resource language results.

Some non-neural methods outperformed neural models on certain languages with limited data.

Abstract

A broad goal in natural language processing (NLP) is to develop a system that has the capacity to process any natural language. Most systems, however, are developed using data from just one language such as English. The SIGMORPHON 2020 shared task on morphological reinflection aims to investigate systems' ability to generalize across typologically distinct languages, many of which are low resource. Systems were developed using data from 45 languages and just 5 language families, fine-tuned with data from an additional 45 languages and 10 language families (13 in total), and evaluated on all 90 languages. A total of 22 systems (19 neural) from 10 teams were submitted to the task. All four winning systems were neural (two monolingual transformers and two massively multilingual RNN-based models with gated attention). Most teams demonstrate utility of data hallucination and augmentation, ensembles, and multilingual training for low-resource languages. Non-neural learners and manually designed grammars showed competitive and even superior performance on some languages (such as Ingrian, Tajik, Tagalog, Zarma, Lingala), especially with very limited data. Some language families (Afro-Asiatic, Niger-Congo, Turkic) were relatively easy for most systems and achieved over 90% mean accuracy while others were more challenging.