A Weakly Supervised Approach for Classifying Stance in Twitter Replies

TL;DR

This paper introduces a weakly supervised method for classifying stance in Twitter replies, leveraging hashtags for labeling and outperforming supervised models without needing extensive labeled data.

Contribution

The study presents a novel weakly supervised approach for stance classification that reduces reliance on labeled data by using hashtags for weak labeling.

Findings

Improves mean F1-macro by 8% over supervised models

Creates a new stance dataset with user opinions and stances

Demonstrates effectiveness on COVID-19 Twitter conversations

Abstract

Conversations on social media (SM) are increasingly being used to investigate social issues on the web, such as online harassment and rumor spread. For such issues, a common thread of research uses adversarial reactions, e.g., replies pointing out factual inaccuracies in rumors. Though adversarial reactions are prevalent in online conversations, inferring those adverse views (or stance) from the text in replies is difficult and requires complex natural language processing (NLP) models. Moreover, conventional NLP models for stance mining need labeled data for supervised learning. Getting labeled conversations can itself be challenging as conversations can be on any topic, and topics change over time. These challenges make learning the stance a difficult NLP problem. In this research, we first create a new stance dataset comprised of three different topics by labeling both users' opinions on the topics (as in pro/con) and users' stance while replying to others' posts (as in favor/oppose). As we find limitations with supervised approaches, we propose a weakly-supervised approach to predict the stance in Twitter replies. Our novel method allows using a smaller number of hashtags to generate weak labels for Twitter replies. Compared to supervised learning, our method improves the mean F1-macro by 8\% on the hand-labeled dataset without using any hand-labeled examples in the training set. We further show the applicability of our proposed method on COVID 19 related conversations on Twitter.