Detecting Abusive Albanian

TL;DR

This paper introduces extsc{Shaj}, an annotated Albanian dataset for hate and offensive speech detection, addressing language gaps in social media content moderation research, and evaluates classification models on this dataset.

Contribution

The paper provides the first annotated Albanian dataset for hate speech detection and evaluates multiple models, advancing NLP resources for underrepresented languages.

Findings

Best model achieved 0.77 F1 for offensive language detection

Achieved 0.64 F1 for offensive type categorization

Achieved 0.52 F1 for offensive target identification

Abstract

The ever growing usage of social media in the recent years has had a direct impact on the increased presence of hate speech and offensive speech in online platforms. Research on effective detection of such content has mainly focused on English and a few other widespread languages, while the leftover majority fail to have the same work put into them and thus cannot benefit from the steady advancements made in the field. In this paper we present \textsc{Shaj}, an annotated Albanian dataset for hate speech and offensive speech that has been constructed from user-generated content on various social media platforms. Its annotation follows the hierarchical schema introduced in OffensEval. The dataset is tested using three different classification models, the best of which achieves an F1 score of 0.77 for the identification of offensive language, 0.64 F1 score for the automatic categorization of offensive types and lastly, 0.52 F1 score for the offensive language target identification.