An Accurate Arabic Root-Based Lemmatizer for Information Retrieval Purposes

TL;DR

This paper introduces the first accurate, non-statistical Arabic lemmatizer tailored for information retrieval, utilizing language resources to improve lemma accuracy and outperform existing models.

Contribution

It presents a novel Arabic lemmatizer algorithm that leverages language resources, achieving higher accuracy than current models for IR applications.

Findings

Maximum accuracy of 94.8% in POS tagging.

89.15% accuracy on first seen documents, outperforming Stanford model.

Significant improvement in Arabic lemma accuracy for IR.

Abstract

In spite of its robust syntax, semantic cohesion, and less ambiguity, lemma level analysis and generation does not yet focused in Arabic NLP literatures. In the current research, we propose the first non-statistical accurate Arabic lemmatizer algorithm that is suitable for information retrieval (IR) systems. The proposed lemmatizer makes use of different Arabic language knowledge resources to generate accurate lemma form and its relevant features that support IR purposes. As a POS tagger, the experimental results show that, the proposed algorithm achieves a maximum accuracy of 94.8%. For first seen documents, an accuracy of 89.15% is achieved, compared to 76.7% of up to date Stanford accurate Arabic model, for the same, dataset.