SocialNLP Fake-EmoReact 2021 Challenge Overview: Predicting Fake Tweets from Their Replies and GIFs

TL;DR

This paper overviews the Fake-EmoReact 2021 Challenge focused on predicting tweet authenticity using reply context and GIF data, introducing a large dataset and analyzing team approaches and results.

Contribution

It introduces the Fake-EmoReact dataset with over 453,000 labeled tweets and provides a comprehensive overview of the challenge, participant methods, and performance outcomes.

Findings

Best model achieved 93.9 F1 score

24 teams registered, 5 submitted results

Diverse approaches analyzed for fake tweet detection

Abstract

This paper provides an overview of the Fake-EmoReact 2021 Challenge, held at the 9th SocialNLP Workshop, in conjunction with NAACL 2021. The challenge requires predicting the authenticity of tweets using reply context and augmented GIF categories from EmotionGIF dataset. We offer the Fake-EmoReact dataset with more than 453k as the experimental materials, where every tweet is labeled with authenticity. Twenty-four teams registered to participate in this challenge, and 5 submitted their results successfully in the evaluation phase. The best team achieves 93.9 on Fake-EmoReact 2021 dataset using F1 score. In addition, we show the definition of share task, data collection, and the teams' performance that joined this challenge and their approaches.