Findings of the NLP4IF-2019 Shared Task on Fine-Grained Propaganda Detection

TL;DR

This paper reports on the NLP4IF-2019 shared task focused on detecting propaganda in news articles, involving fragment-level and sentence-level classification tasks, with multiple teams achieving significant improvements over baselines.

Contribution

It introduces a shared task on fine-grained propaganda detection, providing datasets, benchmarks, and a competitive platform for system development.

Findings

Most systems outperformed baselines significantly

12 teams participated in fragment-level task

25 teams participated in sentence-level task

Abstract

We present the shared task on Fine-Grained Propaganda Detection, which was organized as part of the NLP4IF workshop at EMNLP-IJCNLP 2019. There were two subtasks. FLC is a fragment-level task that asks for the identification of propagandist text fragments in a news article and also for the prediction of the specific propaganda technique used in each such fragment (18-way classification task). SLC is a sentence-level binary classification task asking to detect the sentences that contain propaganda. A total of 12 teams submitted systems for the FLC task, 25 teams did so for the SLC task, and 14 teams eventually submitted a system description paper. For both subtasks, most systems managed to beat the baseline by a sizable margin. The leaderboard and the data from the competition are available at http://propaganda.qcri.org/nlp4if-shared-task/.