Monolingual and Parallel Corpora for Kangri Low Resource Language

TL;DR

This paper introduces the first comprehensive Kangri language corpus, including monolingual and parallel data, along with pre-trained embeddings and translation evaluation metrics, to support research on this endangered language.

Contribution

It provides the first digital corpus and resources for Kangri, a low-resource endangered language, facilitating future NLP research and language preservation efforts.

Findings

Achieved BLEU and METEOR scores for SMT and NMT models.

Shared pre-trained Kangri word embeddings.

Made the corpus publicly available for research.

Abstract

In this paper we present the dataset of Himachali low resource endangered language, Kangri (ISO 639-3xnr) listed in the United Nations Educational, Scientific and Cultural Organization (UNESCO). The compilation of kangri corpus has been a challenging task due to the non-availability of the digitalized resources. The corpus contains 1,81,552 Monolingual and 27,362 Hindi-Kangri Parallel corpora. We shared pre-trained kangri word embeddings. We also reported the Bilingual Evaluation Understudy (BLEU) score and Metric for Evaluation of Translation with Explicit ORdering (METEOR) score of Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) results for the corpus. The corpus is freely available for non-commercial usages and research. To the best of our knowledge, this is the first Himachali low resource endangered language corpus. The resources are available at (https://github.com/chauhanshweta/Kangri_corpus)