Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates

TL;DR

This study analyzes 155 years of German parliamentary debates to understand how solidarity towards women and migrants has evolved, utilizing large language models to automate annotation and reveal shifts in social attitudes over time.

Contribution

The paper introduces a novel fine-grained framework for detecting solidarity in political discourse and demonstrates the effectiveness of GPT-4 in automating large-scale annotation tasks.

Findings

Solidarity with migrants exceeds anti-solidarity over time.

Shift from group-based to compassionate and exchange-based solidarity.

GPT-4 approaches human annotation quality at a lower cost.

Abstract

Solidarity is a crucial concept to understand social relations in societies. In this paper, we explore fine-grained solidarity frames to study solidarity towards women and migrants in German parliamentary debates between 1867 and 2022. Using 2,864 manually annotated text snippets (with a cost exceeding 18k Euro), we evaluate large language models (LLMs) like Llama 3, GPT-3.5, and GPT-4. We find that GPT-4 outperforms other LLMs, approaching human annotation quality. Using GPT-4, we automatically annotate more than 18k further instances (with a cost of around 500 Euro) across 155 years and find that solidarity with migrants outweighs anti-solidarity but that frequencies and solidarity types shift over time. Most importantly, group-based notions of (anti-)solidarity fade in favor of compassionate solidarity, focusing on the vulnerability of migrant groups, and exchange-based anti-solidarity, focusing on the lack of (economic) contribution. Our study highlights the interplay of historical events, socio-economic needs, and political ideologies in shaping migration discourse and social cohesion. We also show that powerful LLMs, if carefully prompted, can be cost-effective alternatives to human annotation for hard social scientific tasks.