Crowd Intelligence for Early Misinformation Prediction on Social Media

TL;DR

This paper presents CROWDSHIELD, a novel crowd intelligence-based method utilizing deep Q-learning and transformer encoders for early misinformation prediction on social media, outperforming baseline systems on a new annotated Twitter dataset.

Contribution

The paper introduces CROWDSHIELD, combining crowd reaction analysis with deep reinforcement learning and content understanding for early misinformation detection, along with a new annotated Twitter dataset.

Findings

CROWDSHIELD achieves ~4% higher macro-F1 score than baselines.

Deep Q-learning effectively captures stance and claim dimensions.

The approach outperforms existing methods in early misinformation prediction.

Abstract

Misinformation spreads rapidly on social media, causing serious damage by influencing public opinion, promoting dangerous behavior, or eroding trust in reliable sources. It spreads too fast for traditional fact-checking, stressing the need for predictive methods. We introduce CROWDSHIELD, a crowd intelligence-based method for early misinformation prediction. We hypothesize that the crowd's reactions to misinformation reveal its accuracy. Furthermore, we hinge upon exaggerated assertions/claims and replies with particular positions/stances on the source post within a conversation thread. We employ Q-learning to capture the two dimensions -- stances and claims. We utilize deep Q-learning due to its proficiency in navigating complex decision spaces and effectively learning network properties. Additionally, we use a transformer-based encoder to develop a comprehensive understanding of both content and context. This multifaceted approach helps ensure the model pays attention to user interaction and stays anchored in the communication's content. We propose MIST, a manually annotated misinformation detection Twitter corpus comprising nearly 200 conversation threads with more than 14K replies. In experiments, CROWDSHIELD outperformed ten baseline systems, achieving an improvement of ~4% macro-F1 score. We conduct an ablation study and error analysis to validate our proposed model's performance. The source code and dataset are available at https://github.com/LCS2-IIITD/CrowdShield.git.