Reasoner Outperforms: Generative Stance Detection with Rationalization for Social Media

TL;DR

This paper introduces a generative stance detection method with rationales that enhances interpretability and outperforms larger models, contributing to trustworthy social media analysis.

Contribution

It presents a novel generative approach with rationales for stance detection, improving interpretability and model performance, especially in smaller language models.

Findings

Incorporating reasoning improves smaller model performance by up to 9.57%.

Reasoning enhances multitask learning but may reduce single-task effectiveness.

Faithful rationales aid in rationale distillation and interpretability.

Abstract

Stance detection is crucial for fostering a human-centric Web by analyzing user-generated content to identify biases and harmful narratives that undermine trust. With the development of Large Language Models (LLMs), existing approaches treat stance detection as a classification problem, providing robust methodologies for modeling complex group interactions and advancing capabilities in natural language tasks. However, these methods often lack interpretability, limiting their ability to offer transparent and understandable justifications for predictions. This study adopts a generative approach, where stance predictions include explicit, interpretable rationales, and integrates them into smaller language models through single-task and multitask learning. We find that incorporating reasoning into stance detection enables the smaller model (FlanT5) to outperform GPT-3.5's zero-shot performance, achieving an improvement of up to 9.57%. Moreover, our results show that reasoning capabilities enhance multitask learning performance but may reduce effectiveness in single-task settings. Crucially, we demonstrate that faithful rationales improve rationale distillation into SLMs, advancing efforts to build interpretable, trustworthy systems for addressing discrimination, fostering trust, and promoting equitable engagement on social media.