Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection

TL;DR

This paper introduces a multilingual dataset for radical content detection, analyzes annotation biases and disagreements, and explores how socio-demographic factors influence annotations and model outcomes, emphasizing fairness and transparency.

Contribution

It provides a new multilingual dataset with detailed annotation analysis and investigates socio-demographic impacts on annotation and model performance.

Findings

Annotation disagreements reveal biases affecting model accuracy.

Synthetic data shows socio-demographic traits influence annotation patterns.

Bias analysis highlights the need for fair and transparent radical content detection models.

Abstract

The proliferation of radical content on online platforms poses significant risks, including inciting violence and spreading extremist ideologies. Despite ongoing research, existing datasets and models often fail to address the complexities of multilingual and diverse data. To bridge this gap, we introduce a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. This dataset is pseudonymized to protect individual privacy while preserving contextual information. Beyond presenting our freely available dataset, we analyze the annotation process, highlighting biases and disagreements among annotators and their implications for model performance. Additionally, we use synthetic data to investigate the influence of socio-demographic traits on annotation patterns and model predictions. Our work offers a comprehensive examination of the challenges and opportunities in building robust datasets for radical content detection, emphasizing the importance of fairness and transparency in model development.