Robust Stance Detection: Understanding Public Perceptions in Social Media

TL;DR

This paper introduces STANCE-C3, a novel framework combining counterfactual data augmentation and contrastive learning to improve the robustness and accuracy of stance detection models across diverse social media domains and topics.

Contribution

The paper presents STANCE-C3, a domain-adaptive stance detection framework that enhances cross-domain performance using counterfactual augmentation and contrastive learning techniques.

Findings

STANCE-C3 outperforms baseline models in accuracy across multiple domains.

The framework demonstrates robustness to domain shifts and topic variations.

Specialized stance detection models are valuable in safety-critical applications.

Abstract

The abundance of social media data has presented opportunities for accurately determining public and group-specific stances around policy proposals or controversial topics. In contrast with sentiment analysis which focuses on identifying prevailing emotions, stance detection identifies precise positions (i.e., supportive, opposing, neutral) relative to a well-defined topic, such as perceptions toward specific global health interventions during the COVID-19 pandemic. Traditional stance detection models, while effective within their specific domain (e.g., attitudes towards masking protocols during COVID-19), often lag in performance when applied to new domains and topics due to changes in data distribution. This limitation is compounded by the scarcity of domain-specific, labeled datasets, which are expensive and labor-intensive to create. A solution we present in this paper combines counterfactual data augmentation with contrastive learning to enhance the robustness of stance detection across domains and topics of interest. We evaluate the performance of current state-of-the-art stance detection models, including a prompt-optimized large language model, relative to our proposed framework succinctly called STANCE-C3 (domain-adaptive Cross-target STANCE detection via Contrastive learning and Counterfactual generation). Empirical evaluations demonstrate STANCE-C3's consistent improvements over the baseline models with respect to accuracy across domains and varying focal topics. Despite the increasing prevalence of general-purpose models such as generative AI, specialized models such as STANCE-C3 provide utility in safety-critical domains wherein precision is highly valued, especially when a nuanced understanding of the concerns of different population segments could result in crafting more impactful public policies.