Arabic Dialect Identification in the Wild

TL;DR

This paper introduces QADI, a large-scale dataset of 540,000 tweets from 18 Arab countries, enabling improved dialect identification with an F1-score of 60.6%, advancing research in Arabic dialect recognition.

Contribution

The paper presents a novel, automatically collected dataset for Arabic dialect identification covering 18 countries, with high label accuracy and effective dialect classification results.

Findings

Dataset contains 540k tweets from 2,525 users across 18 countries.

Achieved 91.5% label accuracy in intrinsic evaluation.

Attained 60.6% macro F1-score in dialect classification.

Abstract

We present QADI, an automatically collected dataset of tweets belonging to a wide range of country-level Arabic dialects -covering 18 different countries in the Middle East and North Africa region. Our method for building this dataset relies on applying multiple filters to identify users who belong to different countries based on their account descriptions and to eliminate tweets that are either written in Modern Standard Arabic or contain inappropriate language. The resultant dataset contains 540k tweets from 2,525 users who are evenly distributed across 18 Arab countries. Using intrinsic evaluation, we show that the labels of a set of randomly selected tweets are 91.5% accurate. For extrinsic evaluation, we are able to build effective country-level dialect identification on tweets with a macro-averaged F1-score of 60.6% across 18 classes.