Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube

TL;DR

This study evaluates YouTube's effectiveness in regulating election misinformation through a crowd-sourced audit, revealing that while most content is appropriately moderated, some misinformative videos still influence viewers.

Contribution

It provides an empirical assessment of YouTube's policy enactment on election misinformation using a novel crowd-sourced browser extension methodology.

Findings

YouTube's search results favor videos opposing misinformation.

Misinformative videos still appear in recommendation trails after viewing.

Overall, YouTube largely succeeds but has room for improvement.

Abstract

With the 2022 US midterm elections approaching, conspiratorial claims about the 2020 presidential elections continue to threaten users' trust in the electoral process. To regulate election misinformation, YouTube introduced policies to remove such content from its searches and recommendations. In this paper, we conduct a 9-day crowd-sourced audit on YouTube to assess the extent of enactment of such policies. We recruited 99 users who installed a browser extension that enabled us to collect up-next recommendation trails and search results for 45 videos and 88 search queries about the 2020 elections. We find that YouTube's search results, irrespective of search query bias, contain more videos that oppose rather than support election misinformation. However, watching misinformative election videos still lead users to a small number of misinformative videos in the up-next trails. Our results imply that while YouTube largely seems successful in regulating election misinformation, there is still room for improvement.