Conspiracy theories and where to find them on TikTok

TL;DR

This study systematically analyzes conspiracy theories on TikTok, quantifying their prevalence, examining the impact of monetization programs, and evaluating AI models for detecting harmful content to aid moderation efforts.

Contribution

It provides the first large-scale longitudinal dataset of TikTok conspiracy videos and assesses the effectiveness of Large Language Models in identifying harmful content.

Findings

Up to 1000 new conspiracy videos per month

Increase in video duration linked to monetization programs

Large Language Models achieve up to 96% precision in detection

Abstract

TikTok has skyrocketed in popularity over recent years, especially among younger audiences. However, there are public concerns about the potential of this platform to promote and amplify harmful content. This study presents the first systematic analysis of conspiracy theories on TikTok. By leveraging the official TikTok Research API we collect a longitudinal dataset of 1.5M videos shared in the U.S. over three years. We estimate a lower bound on the prevalence of conspiratorial videos (up to 1000 new videos per month) and evaluate the effects of TikTok's Creativity Program for monetization, observing an overall increase in video duration regardless of content. Lastly, we evaluate the capabilities of state-of-the-art open-weight Large Language Models to identify conspiracy theories from audio transcriptions of videos. While these models achieve high precision in detecting harmful content (up to 96%), their overall performance remains comparable to fine-tuned traditional models such as RoBERTa. Our findings suggest that Large Language Models can serve as an effective tool for supporting content moderation strategies aimed at reducing the spread of harmful content on TikTok.