EENLP: Cross-lingual Eastern European NLP Index

TL;DR

This paper introduces EENLP, a comprehensive index of Eastern European NLP resources and datasets for semantic tasks, aiming to address resource scarcity and establish performance baselines for multilingual models.

Contribution

It provides the first extensive index of Eastern European NLP datasets and models, along with cross-lingual semantic datasets for evaluation.

Findings

Multilingual models show varying performance across Eastern European languages.

The index facilitates resource discovery and community collaboration.

Baseline results highlight gaps and opportunities for future research.

Abstract

Motivated by the sparsity of NLP resources for Eastern European languages, we present a broad index of existing Eastern European language resources (90+ datasets and 45+ models) published as a github repository open for updates from the community. Furthermore, to support the evaluation of commonsense reasoning tasks, we provide hand-crafted cross-lingual datasets for five different semantic tasks (namely news categorization, paraphrase detection, Natural Language Inference (NLI) task, tweet sentiment detection, and news sentiment detection) for some of the Eastern European languages. We perform several experiments with the existing multilingual models on these datasets to define the performance baselines and compare them to the existing results for other languages.