Community Notes vs. Snoping: How the Crowd Selects Fact-Checking Targets on Social Media

TL;DR

This study compares community-based fact-checking on Twitter with traditional snoping, revealing distinct target selection, speed, and agreement levels, highlighting the importance of multiple approaches in misinformation research.

Contribution

It provides the first detailed comparison of community notes and snoping, showing their different target focuses, speeds, and agreement levels in social media fact-checking.

Findings

Community Notes target larger, more influential accounts.

Snopers fact-check at higher speed.

High agreement in overlapping fact-checks.

Abstract

Deploying links to fact-checking websites (so-called "snoping") is a common intervention that can be used by social media users to refute misleading claims. However, its real-world effect may be limited as it suffers from low visibility and distrust towards professional fact-checkers. As a remedy, Twitter launched its community-based fact-checking system Community Notes on which fact-checks are carried out by actual Twitter users and directly shown on the fact-checked tweets. Yet, an understanding of how fact-checking via Community Notes differs from snoping is absent. In this study, we analyze differences in how contributors to Community Notes and Snopers select their targets when fact-checking social media posts. For this purpose, we analyze two unique datasets from Twitter: (a) 25,912 community-created fact-checks from Twitter's Community Notes platform; and (b) 52,505 "snopes" that debunk tweets via fact-checking replies linking to professional fact-checking websites. We find that Notes contributors and Snopers focus on different targets when fact-checking social media content. For instance, Notes contributors tend to fact-check posts from larger accounts with higher social influence and are relatively less likely to endorse/emphasize the accuracy of not misleading posts. Fact-checking targets of Notes contributors and Snopers rarely overlap; however, those overlapping exhibit a high level of agreement in the fact-checking assessment. Moreover, we demonstrate that Snopers fact-check social media posts at a higher speed. Altogether, our findings imply that different fact-checking approaches -- carried out on the same social media platform -- can result in vastly different social media posts getting fact-checked. This has important implications for future research on misinformation, which should not rely on a single fact-checking approach when compiling misinformation datasets.