Findings of the NLP4IF-2021 Shared Tasks on Fighting the COVID-19 Infodemic and Censorship Detection

TL;DR

This paper reports on the NLP4IF-2021 shared tasks which focused on combating COVID-19 misinformation and censorship detection, highlighting system performances, methods used, and key findings across multiple languages.

Contribution

It introduces two new shared tasks on COVID-19 infodemic and censorship detection, providing datasets, evaluation metrics, and analyzing system approaches and results.

Findings

Most systems improved over baselines

Pre-trained Transformers and ensembles were most effective

Task datasets and leaderboards are publicly available

Abstract

We present the results and the main findings of the NLP4IF-2021 shared tasks. Task 1 focused on fighting the COVID-19 infodemic in social media, and it was offered in Arabic, Bulgarian, and English. Given a tweet, it asked to predict whether that tweet contains a verifiable claim, and if so, whether it is likely to be false, is of general interest, is likely to be harmful, and is worthy of manual fact-checking; also, whether it is harmful to society, and whether it requires the attention of policy makers. Task~2 focused on censorship detection, and was offered in Chinese. A total of ten teams submitted systems for task 1, and one team participated in task 2; nine teams also submitted a system description paper. Here, we present the tasks, analyze the results, and discuss the system submissions and the methods they used. Most submissions achieved sizable improvements over several baselines, and the best systems used pre-trained Transformers and ensembles. The data, the scorers and the leaderboards for the tasks are available at http://gitlab.com/NLP4IF/nlp4if-2021.