The FIGNEWS Shared Task on News Media Narratives

TL;DR

The FIGNEWS shared task organized at ArabicNLP 2024 aimed to develop multilingual annotation frameworks for bias and propaganda in news media, involving diverse languages and extensive participant collaboration.

Contribution

This work introduces a multilingual shared task focused on bias and propaganda annotation, creating new guidelines and datasets across five languages during a significant geopolitical event.

Findings

129,800 data points collected

High participation across multiple languages

Insights into annotation quality and consistency

Abstract

We present an overview of the FIGNEWS shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. The shared task addresses bias and propaganda annotation in multilingual news posts. We focus on the early days of the Israel War on Gaza as a case study. The task aims to foster collaboration in developing annotation guidelines for subjective tasks by creating frameworks for analyzing diverse narratives highlighting potential bias and propaganda. In a spirit of fostering and encouraging diversity, we address the problem from a multilingual perspective, namely within five languages: English, French, Arabic, Hebrew, and Hindi. A total of 17 teams participated in two annotation subtasks: bias (16 teams) and propaganda (6 teams). The teams competed in four evaluation tracks: guidelines development, annotation quality, annotation quantity, and consistency. Collectively, the teams produced 129,800 data points. Key findings and implications for the field are discussed.