Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models

TL;DR

This paper demonstrates how large language models can effectively detect hope, hate, and emotions in Arabic text and memes, achieving high accuracy and winning the Mahed 2025 challenge.

Contribution

It evaluates and fine-tunes large language models for Arabic content analysis, achieving state-of-the-art performance in hate speech and emotion detection in text and memes.

Findings

LLMs like GPT-4o-mini and Gemini Flash 2.5 outperform baselines.

Fine-tuned models achieve up to 79.6% macro F1 score.

The approach secures first place in the Mahed 2025 challenge.

Abstract

The rise of social media and online communication platforms has led to the spread of Arabic textual posts and memes as a key form of digital expression. While these contents can be humorous and informative, they are also increasingly being used to spread offensive language and hate speech. Consequently, there is a growing demand for precise analysis of content in Arabic text and memes. This paper explores the potential of large language models to effectively identify hope, hate speech, offensive language, and emotional expressions within such content. We evaluate the performance of base LLMs, fine-tuned LLMs, and pre-trained embedding models. The evaluation is conducted using a dataset of Arabic textual speech and memes proposed in the ArabicNLP MAHED 2025 challenge. The results underscore the capacity of LLMs such as GPT-4o-mini, fine-tuned with Arabic textual speech, and Gemini Flash 2.5, fine-tuned with Arabic memes, to deliver the superior performance. They achieve up to 72.1%, 57.8%, and 79.6% macro F1 scores for tasks 1, 2, and 3, respectively, and secure first place overall in the Mahed 2025 challenge. The proposed solutions offer a more nuanced understanding of both text and memes for accurate and efficient Arabic content moderation systems.