The SIGMORPHON 2020 Shared Task on Unsupervised Morphological Paradigm Completion

TL;DR

This paper reports on the SIGMORPHON 2020 shared task focused on unsupervised morphological paradigm completion, revealing that current systems struggle to outperform simple baselines across multiple languages, highlighting the task's difficulty.

Contribution

It introduces a new unsupervised task for morphological paradigm completion and provides a benchmark with baseline systems, setting a foundation for future research in the area.

Findings

No submitted system outperformed the baseline on average across all languages.

Only 3 languages saw the best results from submitted systems.

Unsupervised morphological paradigm completion remains largely unsolved.

Abstract

In this paper, we describe the findings of the SIGMORPHON 2020 shared task on unsupervised morphological paradigm completion (SIGMORPHON 2020 Task 2), a novel task in the field of inflectional morphology. Participants were asked to submit systems which take raw text and a list of lemmas as input, and output all inflected forms, i.e., the entire morphological paradigm, of each lemma. In order to simulate a realistic use case, we first released data for 5 development languages. However, systems were officially evaluated on 9 surprise languages, which were only revealed a few days before the submission deadline. We provided a modular baseline system, which is a pipeline of 4 components. 3 teams submitted a total of 7 systems, but, surprisingly, none of the submitted systems was able to improve over the baseline on average over all 9 test languages. Only on 3 languages did a submitted system obtain the best results. This shows that unsupervised morphological paradigm completion is still largely unsolved. We present an analysis here, so that this shared task will ground further research on the topic.