Investigating Machine Learning Methods for Language and Dialect Identification of Cuneiform Texts

TL;DR

This paper explores machine learning techniques for identifying seven cuneiform languages and dialects, addressing resource scarcity and tokenization challenges, with ensemble classifiers achieving a 72.10% F1-score.

Contribution

It presents the first application of machine learning methods to cuneiform language identification, demonstrating effective ensemble models on this low-resource script.

Findings

Ensemble of SVM and naive Bayes classifiers achieved 72.10% F1-score.

Character-level features are effective for cuneiform language identification.

The study advances computational methods for ancient script analysis.

Abstract

Identification of the languages written using cuneiform symbols is a difficult task due to the lack of resources and the problem of tokenization. The Cuneiform Language Identification task in VarDial 2019 addresses the problem of identifying seven languages and dialects written in cuneiform; Sumerian and six dialects of Akkadian language: Old Babylonian, Middle Babylonian Peripheral, Standard Babylonian, Neo-Babylonian, Late Babylonian, and Neo-Assyrian. This paper describes the approaches taken by SharifCL team to this problem in VarDial 2019. The best result belongs to an ensemble of Support Vector Machines and a naive Bayes classifier, both working on character-level features, with macro-averaged F1-score of 72.10%.