L(u)PIN: LLM-based Political Ideology Nowcasting

TL;DR

This paper introduces L(u)PIN, a novel method leveraging large language models to nowcast individual politicians' ideological positions by analyzing speech data and projecting embeddings on a chosen ideological axis.

Contribution

It presents a flexible approach using LLMs and fine-tuned BERT classifiers to evaluate individual political stances on specific topics, overcoming data limitations of previous methods.

Findings

Effective in analyzing individual politicians' ideological positions

Utilizes GPT-4 generated reference sentences for flexible stance evaluation

Achieves nuanced, topic-specific political stance measurement

Abstract

The quantitative analysis of political ideological positions is a difficult task. In the past, various literature focused on parliamentary voting data of politicians, party manifestos and parliamentary speech to estimate political disagreement and polarization in various political systems. However previous methods of quantitative political analysis suffered from a common challenge which was the amount of data available for analysis. Also previous methods frequently focused on a more general analysis of politics such as overall polarization of the parliament or party-wide political ideological positions. In this paper, we present a method to analyze ideological positions of individual parliamentary representatives by leveraging the latent knowledge of LLMs. The method allows us to evaluate the stance of politicians on an axis of our choice allowing us to flexibly measure the stance of politicians in regards to a topic/controversy of our choice. We achieve this by using a fine-tuned BERT classifier to extract the opinion-based sentences from the speeches of representatives and projecting the average BERT embeddings for each representative on a pair of reference seeds. These reference seeds are either manually chosen representatives known to have opposing views on a particular topic or they are generated sentences which where created using the GPT-4 model of OpenAI. We created the sentences by prompting the GPT-4 model to generate a speech that would come from a politician defending a particular position.