CKMorph: A Comprehensive Morphological Analyzer for Central Kurdish

TL;DR

CKMorph is a comprehensive morphological analyzer for Central Kurdish that leverages rule-based methods and a large lexicon, achieving high accuracy and coverage, and providing valuable resources for future NLP research in the language.

Contribution

This paper introduces CKMorph, the first comprehensive rule-based morphological analyzer for Central Kurdish, including a large lexicon, rule set, and benchmark datasets.

Findings

Achieved 95.9% accuracy on the test set.

Covered 95.5% of 4.22 million tokens.

Resources are publicly available for future research.

Abstract

A morphological analyzer, which is a significant component of many natural language processing applications especially for morphologically rich languages, divides an input word into all its composing morphemes and identifies their morphological roles. In this paper, we introduce a comprehensive morphological analyzer for Central Kurdish (CK), a low-resourced language with a rich morphology. Building upon the limited existing literature, we first assembled and systematically categorized a comprehensive collection of the morphological and morphophonological rules of the language. Additionally, we collected and manually labeled a generative lexicon containing nearly 10,000 verb, noun and adjective stems, named entities, and other types of word stems. We used these rule sets and resources to implement CKMorph Analyzer based on finite-state transducers. In order to provide a benchmark for future research, we collected, manually labeled, and publicly shared test sets for evaluating accuracy and coverage of the analyzer. CKMorph was able to correctly analyze 95.9% of the accuracy test set, containing 1,000 CK words morphologically analyzed according to the context. Moreover, CKMorph gave at least one analysis for 95.5% of 4.22M CK tokens of the coverage test set. The demonstration of the application and resources including CK verb database and test sets are openly accessible at https://github.com/CKMorph.