dzStance at StanceEval2024: Arabic Stance Detection based on Sentence Transformers

TL;DR

This paper evaluates the effectiveness of Sentence Transformers versus TF-IDF features for stance detection in Arabic, demonstrating superior performance and competitive results in a shared task across three societal topics.

Contribution

It introduces a comparison between Sentence Transformers and TF-IDF for stance detection, showing the former's superior performance in an Arabic context.

Findings

Sentence Transformers outperform TF-IDF in stance detection accuracy.

The approach achieved top-15 rankings in a competitive shared task.

Promising F1-scores indicate effectiveness for societal issue analysis.

Abstract

This study compares Term Frequency-Inverse Document Frequency (TF-IDF) features with Sentence Transformers for detecting writers' stances--favorable, opposing, or neutral--towards three significant topics: COVID-19 vaccine, digital transformation, and women empowerment. Through empirical evaluation, we demonstrate that Sentence Transformers outperform TF-IDF features across various experimental setups. Our team, dzStance, participated in a stance detection competition, achieving the 13th position (74.91%) among 15 teams in Women Empowerment, 10th (73.43%) in COVID Vaccine, and 12th (66.97%) in Digital Transformation. Overall, our team's performance ranked 13th (71.77%) among all participants. Notably, our approach achieved promising F1-scores, highlighting its effectiveness in identifying writers' stances on diverse topics. These results underscore the potential of Sentence Transformers to enhance stance detection models for addressing critical societal issues.