Large Language Models Unpack Complex Political Opinions through Target-Stance Extraction

TL;DR

This paper demonstrates that large language models can effectively extract detailed target and stance information from complex political discussions on Reddit, matching human performance with minimal supervision.

Contribution

It introduces Target-Stance Extraction (TSE) for nuanced political opinion analysis and evaluates LLMs' effectiveness in this task using a new Reddit dataset.

Findings

LLMs perform comparably to trained human annotators.

Models remain robust on challenging posts with low inter-annotator agreement.

Zero-shot and few-shot prompting strategies are effective for TSE.

Abstract

Political polarization emerges from a complex interplay of beliefs about policies, figures, and issues. However, most computational analyses reduce discourse to coarse partisan labels, overlooking how these beliefs interact. This is especially evident in online political conversations, which are often nuanced and cover a wide range of subjects, making it difficult to automatically identify the target of discussion and the opinion expressed toward them. In this study, we investigate whether Large Language Models (LLMs) can address this challenge through Target-Stance Extraction (TSE), a recent natural language processing task that combines target identification and stance detection, enabling more granular analysis of political opinions. For this, we construct a dataset of 1,084 Reddit posts from r/NeutralPolitics, covering 138 distinct political targets and evaluate a range of proprietary and open-source LLMs using zero-shot, few-shot, and context-augmented prompting strategies. Our results show that the best models perform comparably to highly trained human annotators and remain robust on challenging posts with low inter-annotator agreement. These findings demonstrate that LLMs can extract complex political opinions with minimal supervision, offering a scalable tool for computational social science and political text analysis.