Language Models Learn Metadata: Political Stance Detection Case Study

TL;DR

This paper shows that simple metadata, like party membership, can significantly improve political stance detection, outperforming complex methods, and highlights the importance of how metadata is incorporated.

Contribution

It reveals that straightforward metadata usage can surpass complex models in political stance detection and emphasizes optimal metadata integration strategies.

Findings

Party membership alone outperforms previous methods.

Prepending metadata to speeches yields best results.

Complex metadata systems may not learn effectively.

Abstract

Stance detection is a crucial NLP task with numerous applications in social science, from analyzing online discussions to assessing political campaigns. This paper investigates the optimal way to incorporate metadata into a political stance detection task. We demonstrate that previous methods combining metadata with language-based data for political stance detection have not fully utilized the metadata information; our simple baseline, using only party membership information, surpasses the current state-of-the-art. We then show that prepending metadata (e.g., party and policy) to political speeches performs best, outperforming all baselines, indicating that complex metadata inclusion systems may not learn the task optimally.