Computational Linguistics Meets Libyan Dialect: A Study on Dialect Identification

TL;DR

This paper evaluates various machine learning models for classifying Libyan dialects from Twitter data, finding that Multinomial Naive Bayes with specific n-gram features achieves the highest accuracy of 85.89%, providing empirical benchmarks for Arabic dialect identification.

Contribution

It introduces an empirical comparison of classifiers and feature representations for Libyan dialect identification, highlighting the effectiveness of Multinomial Naive Bayes with n-gram features.

Findings

Multinomial Naive Bayes achieved 85.89% accuracy.

(1,2) word and (1,5) character n-grams are most effective.

MNB outperformed Logistic Regression and SVM.

Abstract

This study investigates logistic regression, linear support vector machine, multinomial Naive Bayes, and Bernoulli Naive Bayes for classifying Libyan dialect utterances gathered from Twitter. The dataset used is the QADI corpus, which consists of 540,000 sentences across 18 Arabic dialects. Preprocessing challenges include handling inconsistent orthographic variations and non-standard spellings typical of the Libyan dialect. The chi-square analysis revealed that certain features, such as email mentions and emotion indicators, were not significantly associated with dialect classification and were thus excluded from further analysis. Two main experiments were conducted: (1) evaluating the significance of meta-features extracted from the corpus using the chi-square test and (2) assessing classifier performance using different word and character n-gram representations. The classification experiments showed that Multinomial Naive Bayes (MNB) achieved the highest accuracy of 85.89% and an F1-score of 0.85741 when using a (1,2) word n-gram and (1,5) character n-gram representation. In contrast, Logistic Regression and Linear SVM exhibited slightly lower performance, with maximum accuracies of 84.41% and 84.73%, respectively. Additional evaluation metrics, including log loss, Cohen kappa, and Matthew correlation coefficient, further supported the effectiveness of MNB in this task. The results indicate that carefully selected n-gram representations and classification models play a crucial role in improving the accuracy of Libyan dialect identification. This study provides empirical benchmarks and insights for future research in Arabic dialect NLP applications.