Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered

TL;DR

This paper introduces neural models for morphological tasks across 22 languages, including endangered ones, using automatically extracted training data from FSTs, and provides resources for further research.

Contribution

It presents a novel method for automatically creating large training datasets from FSTs for multiple languages, including endangered ones, and aligns neural models with existing FST tagsets.

Findings

Neural models achieve effective morphological analysis and generation.

Large datasets are automatically extracted for 22 languages, 17 of which are endangered.

Resources including code, models, and datasets are publicly released.

Abstract

We train neural models for morphological analysis, generation and lemmatization for morphologically rich languages. We present a method for automatically extracting substantially large amount of training data from FSTs for 22 languages, out of which 17 are endangered. The neural models follow the same tagset as the FSTs in order to make it possible to use them as fallback systems together with the FSTs. The source code, models and datasets have been released on Zenodo.