Extracting Participation in Collective Action from Social Media

TL;DR

This paper introduces a set of text classifiers that identify and categorize expressions of participation in collective action on social media, enabling large-scale analysis of online mobilization efforts.

Contribution

It presents a novel, topic-agnostic classification framework trained on Reddit data, capable of detecting nuanced levels of collective action participation using smaller language models.

Findings

Smaller models reliably detect participation (F1=0.71)

Models rival larger ones in capturing participation nuances

Framework outperforms traditional topic modeling and keyword methods

Abstract

Social media play a key role in mobilizing collective action, holding the potential for studying the pathways that lead individuals to actively engage in addressing global challenges. However, quantitative research in this area has been limited by the absence of granular and large-scale ground truth about the level of participation in collective action among individual social media users. To address this limitation, we present a novel suite of text classifiers designed to identify expressions of participation in collective action from social media posts, in a topic-agnostic fashion. Grounded in the theoretical framework of social movement mobilization, our classification captures participation and categorizes it into four levels: recognizing collective issues, engaging in calls-to-action, expressing intention of action, and reporting active involvement. We constructed a labeled training dataset of Reddit comments through crowdsourcing, which we used to train BERT classifiers and fine-tune Llama3 models. Our findings show that smaller language models can reliably detect expressions of participation (weighted F1=0.71), and rival larger models in capturing nuanced levels of participation. By applying our methodology to Reddit, we illustrate its effectiveness as a robust tool for characterizing online communities in innovative ways compared to topic modeling, stance detection, and keyword-based methods. Our framework contributes to Computational Social Science research by providing a new source of reliable annotations useful for investigating the social dynamics of collective action.