Classifying Problem and Solution Framing in Congressional Social Media

TL;DR

This study develops an automated classifier using BERTweet to distinguish problem and solution frames in US Senator social media posts, achieving high accuracy.

Contribution

It introduces a supervised learning approach with BERTweet for framing classification in political social media data, validated on a large dataset.

Findings

Achieved an average weighted F1 score above 0.8 on validation.

Used a dataset of 3,967 labeled tweets with expert annotations.

Demonstrated effective model performance for political communication analysis.

Abstract

Policy setting in the USA according to the ``Garbage Can'' model differentiates between ``problem'' and ``solution'' focused processes. In this paper, we study a large dataset of US Senator postings on Twitter (1.68m tweets in total). Our objective is to develop an automated method to label Senatorial posts as either in the problem or solution streams. Two academic policy experts labeled a subset of 3967 tweets as either problem, solution, or other (anything not problem or solution). We split off a subset of 500 tweets into a test set, with the remaining 3467 used for training. During development, this training set was further split by 60/20/20 proportions for fitting, validation, and development test sets. We investigated supervised learning methods for building problem/solution classifiers directly on the training set, evaluating their performance in terms of F1 score on the validation set, allowing us to rapidly iterate through models and hyperparameters, achieving an average weighted F1 score of above 0.8 on cross validation across the three categories using a BERTweet Base model.