Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification

TL;DR

Arap-Tweet is a comprehensive multi-dialect Arabic Twitter corpus annotated for age, gender, and dialect, aimed at advancing NLP and author profiling for Arabic dialects.

Contribution

This paper introduces Arap-Tweet, a large annotated corpus of Arabic tweets covering multiple dialects, with detailed annotation guidelines and validation, filling a resource gap for Arabic NLP.

Findings

Corpus covers 11 regions and 16 countries.

Annotations include age, gender, and dialect.

Evaluation confirms annotation consistency.

Abstract

In this paper, we present Arap-Tweet, which is a large-scale and multi-dialectal corpus of Tweets from 11 regions and 16 countries in the Arab world representing the major Arabic dialectal varieties. To build this corpus, we collected data from Twitter and we provided a team of experienced annotators with annotation guidelines that they used to annotate the corpus for age categories, gender, and dialectal variety. During the data collection effort, we based our search on distinctive keywords that are specific to the different Arabic dialects and we also validated the location using Twitter API. In this paper, we report on the corpus data collection and annotation efforts. We also present some issues that we encountered during these phases. Then, we present the results of the evaluation performed to ensure the consistency of the annotation. The provided corpus will enrich the limited set of available language resources for Arabic and will be an invaluable enabler for developing author profiling tools and NLP tools for Arabic.